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Abstract 

The  Air  Force  faces  significant  fiscal  challenges  in  the  coming  years.  The  aircraft 
maintenance  depot  activities  at  Ogden  ALC,  Oklahoma  City  ALC  and  Warner- Robins 
ALC  face  complex  operating  environments  due  to  the  diversity  of  aircraft  or  mission 
design  series  (MDS)  maintained  by  each  depot  and  the  variability  of  maintenance 
requirements  for  each  MDS.  Further  complicating  their  operations  is  the  variability  of 
maintenance  actions  required  from  one  aircraft  to  another  within  each  MDS  and  a  highly 
specialized  workforce  that  has  inherent  inflexibility  to  compensate  for  the  workload 
variability.  Air  Force  Materiel  Command  is  reviewing  maintenance  personnel  multi¬ 
skilling  as  a  method  to  efficiently  absorb  the  variability  of  workload  and  maintenance 
requirements  between  aircraft. 

This  research  conducts  an  objective  analysis  of  the  F-22  Heavy  Maintenance 
Modification  Program  by  building  a  discrete  event  simulation  in  ARENA  14®  and 
performing  a  series  of  designed  experiments.  The  study  analyzes  whether  using  a  multi- 
skilled  (flexible)  workforce  will  have  an  impact  on  productivity  of  depot  maintenance 
personnel  through  simulation  of  several  multi-  skilling  policies.  The  research  shows  that 
multi-skilling  policies  can  significantly  outperform  overtime-based  production  timelines 
at  less  cost,  even  if  individual  skill  proficiencies  decline. 
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SIMULATING  F-22  HEAVY  MAINTENANCE  AND  MODIFICATIONS 
WORKFORCE  MULTI-SKILLING 


I.  Introduction 

Background 

The  Air  Force  faces  significant  fiscal  challenges  in  the  coming  years  with 
dwindling  defense  spending  and  increasing  procurement  and  sustainability  costs 
associated  with  more  technologically  advanced  weapons  systems  and  an  aging  legacy 
fleet.  The  Air  Force  Sustainment  Center  and  the  three  Air  Logistics  Complexes  (ALCs) 
under  their  purview  are  at  the  forefront  of  the  battle  to  affect  the  cost  curve  for 
sustainment  operations. 

The  aircraft  maintenance  depot  activities  at  Ogden  ALC,  Oklahoma  City  ALC 
and  Warner- Robins  ALC  face  complex  operating  environments  due  to  the  diversity  of 
aircraft  or  mission  design  series  (MDS)  maintained  by  each  depot  and  the  variability  of 
maintenance  requirements  for  each  MDS.  Further  complicating  their  operations  is  the 
variability  of  maintenance  actions  required  from  one  aircraft  to  another  within  each  MDS 
and  a  highly  specialized  workforce  that  has  inherent  inflexibility  to  compensate  for  the 
workload  variability. 

Multi-skilling  is  one  proposal  that  Air  Force  Materiel  Command  is  reviewing  to 
absorb  the  variability  of  workload  and  maintenance  requirements  between  aircraft. 
Multi-skilling  is  defined  as  “a  position  that  combines  two  or  more  journeyman,  full 
performance  or  higher  level  skills  in  the  same  pay  plan  in  which  formal  on-the-job  or 
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classroom  training  is  required”  (Federal  Service  Impasse  Panel,  1997).  In  1993,  the 
Oklahoma  City  ALC  implemented  a  multi-skill  program,  an  initiative  similar  to  private 
aircraft  and  manufacturing  industries’  directional  shift  towards  more  flexible  workforces 
(Federal  Service  Impasse  Panel,  1997).  The  program  never  materialized  in  the  way 
designed  and  therefore  had  negligible  results.  According  to  a  report  by  the  Air  Force 
Journal  of  Logistics  (2003),  the  benefits  of  multi-skilling  in  the  program  were  not 
realized  due  to  lack  of  supervisory  understanding  on  how  to  use  multi-skill  employees 
and  lack  of  incentives  for  personnel  to  become  multi- skilled.  The  question  remains 
unanswered  on  whether  Air  Force  depot  operations  can  more  cost  effectively  use  their 
work  force  through  multi-skilling. 

Research  Focus 

The  goal  of  this  research  is  to  provide  a  quantitative  analysis  on  whether  a  multi- 
skilled  workforce  allows  for  more  cost  effective  use  of  ALC  maintenance  personnel  The 
focus  of  this  research  is  not  on  the  shortcomings  of  previous  multi-skilling  efforts  or 
methods  for  implement  multi-skilling  but  on  the  potential  benefits  of  a  more  flexible 
workforce. 

Specifically,  AFMC/A4D  requests  this  research  to  focus  on  the  F-22  Heavy 
Maintenance  Modification  Program  at  Ogden  ALC  and  estimate  whether  their  operations 
will  benefit  from  a  more  flexible,  multi- skilled  workforce. 

Proble  m  State  ment 

The  F-22  Heavy  Maintenance  Modification  Program  consists  of  several  Federal 
Wage  Series  occupations  or  maintenance  specialties  that  perform  maintenance  on  the 
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F-22  depot  production  line.  According  to  a  business  case  analysis  (BCA)  on  a  Multi- 
Trade  Demonstration  project  completed  by  Ogden  ALC  (2012),  the  F-22  flight  is 
currently  experiencing  indirect  labor  rates  over  50  percent  and  direct  overtime  rates  over 
15  percent.  The  overtime  hours  equate  to  a  projected  $9.8  million  dollars  in  overtime 
costs  for  the  5-year  period  of2013  to  2017  (Ogden  Air  Logistics  Center,  2012).  The 
business  case  analysis  hypothesizes  that  the  problems  lies  in  the  WG-4102  Painter  (Low 
Observable  or  LO)  workforce  constraints  and  associated  downtime  of  other  occupations 
awaiting  LO  task  completion.  Their  report  identifies  that  LO  coating  related  man-hour 
requirements  account  for  60  percent  of  the  total  man-hour  requirements  (Ogden  Air 
Logistics  Center,  2012). 

The  cost  associated  with  overtime  and  idle  personnel  created  by  the  LO  labor- 
hour  constraint  appears  to  create  an  opportunity  for  potential  productivity  gains  through 
multi-skill  initiatives  by  increasing  the  availability  of  WG-4102  Painter  labor  hours.  No 
substantive  analysis  exists  on  which  maintenance  specialties  within  F-22  depot  operations 
are  favorable  for  multi-skilling  into  the  LO  specialty.  The  literature  review  for  this 
research  contains  further  analysis  of  the  Multi-Trade  Demonstration  Project  business  case 
analysis. 

Additionally,  no  quantitative  analysis  exists  to  ascertain  the  aircraft  throughput 
and  employee  utilization  impacts  of  multi-skilling  other  than  a  limited  simulation  of  the 
KC- 135  IDOCK  depot  process  by  Levien  (2010).  Will  creating  a  flexible  workforce 
increase  productivity,  aircraft  throughput,  and  decrease  indirect  labor  hours? 
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Research  Objectives  &  Questions: 


The  research  will  provide  insight  into  career  fields  that  should  be  considered  for 
multi-skilling  into  LO  and  the  impact  onF-22  throughput  within  the  depot  process. 
Subject  matter  experts  (SMEs)  from  Ogden  Air  Logistics  Complex,  previous  research  in 
queuing  theory,  and  the  research  questions  developed  will  be  the  foundation  for  the 
direction  of  the  research  and  the  desired  output  parameters  selected  for  analysis.  The 
following  research  questions  drive  the  direction  taken  in  this  study. 

Research  Question  1:  Which  maintenance  specialties  within  F-22  Depot  Operations 
should  be  considered  for  multi-skilling  into  the  low  observable  (LO)  specialty? 

Research  Question  2:  How  does  multi-skilling  affect  aircraft  throughput  and  employee 
utilization  in  F-22  Depot  Operations? 

Research  Question  3:  To  what  extent  does  multi-skilling  offset  overtime  use  to  meet 
aircraft  flow  day  requirements  and  throughput  targets? 

Methodology 

The  research  will  conduct  an  objective  analysis  of  the  F-22  Heavy  Maintenance 
Modification  Program  by  building  a  discrete  event  simulation  in  ARENA  14®  and 
performing  a  series  of  designed  experiments.  The  study  analyzes  whether  using  a  multi- 
skilled  workforce  will  have  an  impact  on  productivity  of  depot  maintenance  personnel  by 
simulating  several  multi-skilling  policies. 


4 


Assumptions 


The  only  assumption  at  the  start  of  the  research  is  that  SME  feedback  is  accurate 
and  uninfluenced  by  outside  pressures. 

Implications 

The  implications  of  the  research  could  be  significant  for  Air  Force  Material 
Command.  The  Air  Force  recently  announced  that  the  F-22  Heavy  Maintenance 
Modification  Program  workload  is  going  to  double  within  the  next  12  months  as  depot 
maintenance  activities  previously  performed  at  a  facility  in  Palmdale,  CA  relocate  to 
Ogden  ALC  (United  States  Air  Force,  2013).  The  multi-skilling  analysis  and  results 
from  this  research  could  save  the  Air  Force  millions  of  dollars,  either  by  guiding  to  the 
right  decision  or  by  preventing  the  wrong  one.  Additionally,  decisions  about  future 
weapon  systems  depot  maintenance  specialties  and  their  future  workforce  structures  will 
benefit  from  the  results  of  this  research. 
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II.  Literature  Review 


Chapter  Ove  rview 

The  review  of  literature  for  this  study  encompasses  a  background  on  multi¬ 
skilling,  simulation,  different  analysis  methods,  and  previous  simulation  research  in 
maintenance  and  cross-training  policies.  The  review  starts  by  reviewing  the  proposed 
policy  of  multi-skilling  within  the  United  States  Air  Force. 

Background  on  Multi-skilling 

Multi-skilling  is  previously  defined  as  “a  position  that  combines  two  or  more 
journeyman,  full  performance  or  higher  level  skills  in  the  same  pay  plan  in  which  formal 
on-the-job  or  classroom  training  is  required”  (Federal  Service  Impasse  Panel,  1997).  The 
intent  of  multi-skilling  is  to  buffer  variability  in  depot  maintenance  workloads  by 
providing  a  flexible  workforce  that  is  able  to  work  other  specialty’s  tasks  during  periods 
of  non- utilization  or  to  provide  surge  capacity  during  periods  with  higher  workloads  in 
one  specialty,  effectively  balancing  the  workload. 

The  concept  is  similar  to  flexible  workforce  strategies  adopted  by  commercial 
carriers  and  private  sector  maintenance  repair  organizations  (MROs).  The  commercial 
carriers  and  MROs  hire  Airframe  and  Powerplant  (A&P)  certified  maintenance 
technicians  and  allow  them  to  complete  most  maintenance  tasks  during  major 
maintenance  operations.  Part  of  the  benefit  of  being  A&P  certified  is  that  the  employees 
are  compensated  with  a  higher  wage  for  their  increased  qualifications.  The  Department 
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of  Defense  is  considering  a  similar  incentive  for  multi-skilled  employees  under  a 
proposal  called  multi- trade. 

The  2008  National  Defense  Authorization  Act  authorized  the  Secretary  of  the  Air 
Force  (SEC  AF)  to  carry  out  a  demonstration  project  under  which  workers  who  are 
certified  to  perform  multiple  trades  at  the  journeyman  level  (multi- skilled)  may  be 
promoted  one  grade  level  (Ogden  Air  Logistics  Center,  2012).  The  proposed  project  was 
assigned  to  Ogden  ALC  and  is  expected  to  be  undertaken  on  positions  within  the  F-22 
Heavy  Maintenance  and  Modification  Program  (Ogden  Air  Logistics  Center,  2012).  The 
only  difference  between  the  term  multi-skill  and  multi- trade  is  that  a  multi- skilled 
employee  is  promoted  under  multi-trade  by  one  wage  grade  once  they  are  certified  at  the 
journeyman  level  to  perform  multiple  trades.  Ogden  ALC  performed  a  BCA  in2012  to 
identify  the  potential  benefits  of  a  multi-skilled  workforce. 

The  BCA  identifies  variations  in  work  requirements  of  single- skilled  personnel  as 
a  factor  in  the  peaks  and  valleys  of  resource  usage  within  the  F-22  depot  process.  The 
analysis  further  identifies  the  WG-4102  Painters  with  specialization  in  LO  surface 
application  as  the  “burning  platform”  for  the  multi-trade  initiative  due  to  the 
disproportional  man-hour  requirements  and  use  ofPainters  (LO)  versus  the  other  Federal 
Wage  Series  (FWS)  specialties  (Ogden  Air  Logistics  Center,  2012).  The  BCA  proposes 
the  multi-trade  initiative  as  a  way  to  offset  overtime  requirements  by  increasing  the 
number  ofpersonnel  available  to  complete  LO  maintenance  and  to  increase  overall 
efficiency  of  the  other  specialties  by  allowing  them  to  work  jobs  that  are  more  critical  in 
the  secondary  specialty.  The  BCA  proposes  multi-skilling  all  non-LO  personnel  into  LO 
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and  all  LO  personnel  into  the  non-LO  specialties.  The  analysis  further  hypothesizes 
significant  cost  savings  from  the  initiative. 

The  cost  savings  proposed  by  the  BCA  includes  several  million  dollars  a  year 
after  the  initial  program  implementation  period  with  continued  savings  in  the  out- years  of 
the  analysis.  The  analysis  further  proposes  that  the  garnered  cost  savings/avoidance  from 
reductions  in  personnel  requirements  will  be  a  direct  result  of  employing  a  more  flexible 
workforce,  thereby  increasing  utilization  of  all  personnel  and  minimizing  overtime 
requirements.  With  an  understanding  of  the  proposed  benefits  from  multi-  sk  ill  ing/mult  i- 
trade  policies,  the  researcher  moves  to  a  review  of  simulation  methods  and  best  practices. 

Simulation 

Defined 

A  simulation  is  the  imitation  of  the  operation  of  a  future  or  real-world  process  or 
system  over  time  (Banks,  Carson  II  S,  Nelson  L,  &  Nicol  M,  2010,  p.  3).  Banks  et  al. 
explain  that  many  real-world  systems  are  so  complex  that  models  of  the  systems  are 
virtually  impossible  to  solve  mathematically  (2010).  Simulation  allows  for  the  study  of 
those  systems  that  are  mathematically  infeasible  otherwise. 

The  benefits  of  simulation  include  having  the  capability  to  simulate  potential 
changes  to  the  system  prior  to  implementation  in  order  to  gain  insights  into  the  impact  of 
those  changes  on  system  performance  (Banks  et  aL,  2010,  p.  3).  Another  benefit  is  the 
ability  to  simulate  systems  in  the  designphase  prior  to  building  the  system  (Banks  et  al., 
2010).  The  uses  for  simulation  are  numerous  but  Banks  et  aL  (2010)  identify  1 1  purposes 
for  simulation  including: 


1)  Simulation  enables  the  study  o£  and  experimentation  with,  the  internal 
interactions  of  a  complex  system  or  of  a  subsystem  within  the  complex 
system 

2)  Informational,  organizational,  and  environmental  changes  can  be  simulated, 
and  the  effect  of  these  alterations  on  the  model’s  behavior  can  be  observed. 

3)  The  knowledge  gained  during  the  designing  of  a  simulation  model  could  be  of 
great  value  toward  suggesting  improvement  in  the  system  under  investigation. 

4)  Changing  simulation  inputs  and  observing  the  resulting  outputs  can  produce 
valuable  insights  about  which  variables  are  the  most  important  and  how 
variables  interact. 

5)  Simulation  can  be  used  as  a  pedagogical  device  to  reinforce  analytic  solution 
methodologies. 

6)  Simulation  can  be  used  to  experiment  with  new  designs  or  policies  before 
implementation,  so  as  to  prepare  for  what  might  happen. 

7)  Simulation  can  be  used  to  verily  analytic  solutions. 

8)  Simulating  different  capabilities  for  a  machine  can  help  determine  its 
requirements. 

9)  Simulation  models  designed  for  training  make  learning  possible,  without  the 
costand  disruption  of  on-the-job  instruction. 

10)  Animation  can  show  a  system  in  simulated  operation  so  that  the  plan  can  be 
visualized. 
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1 1)  A  modern  system  (factory,  wafer  fabrication  plant,  service  organization,  etc.) 
is  so  complex  that  its  internal  interactions  can  be  treated  only  through 
simulation.  ( p.  4) 

Banks  et  aL  (2010)  also  identify  times  when  simulation  is  not  appropriate 
including: 

1)  When  the  problem  can  be  solved  by  common  sense. 

2)  If  the  problems  can  be  solved  analytically. 

3)  If  it  is  less  expansive  to  perform  direct  experiments 

4)  If  the  cost  exceed  the  savings. 

5)  If  the  resources  or  time  are  not  available. 

6)  If  no  data  is  available,  not  even  estimates,  simulation  is  not  advised. 

7)  If  managers  have  unreasonable  expectations 

8)  System  behavior  is  too  complex  or  cannot  be  defined  (human  behavior,  etc) 
(p.  4-6) 

The  reasons  for  and  against  using  simulation  should  be  thoroughly 
reviewed  and  considered  before  deciding  simulation  as  the  desired  methodology.  If 
simulation  is  the  path  selected,  Bank  et  al.  (2010)  provide  12  basic  steps  for  conducting  a 
simulation  study  as  depicted  in  F igure  1 .  The  important  concept  to  notice  is  that  once  the 
problem  is  formulated  and  the  objectives  for  the  project  are  set,  the  process  retraces  steps 
as  you  go  and  the  completion  of  a  step  does  not  mean  the  step  will  not  need  to  be  re- 
accomplished  later  in  development  or  during  analysis.  The  researcher  should  not  put  the 
blinders  on  and  assume  that  because  the  model  was  previously  verified  and  validated  that 
additional  verification  and  validation  is  not  required. 
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Figure  1:  Steps  in  a  Simulation  Study  (Banks  et  al,  2010) 

Model  Fidelity 

According  to  Law  (2005),  a  simulation practictioner  must  determine  the  aspects 
of  a  system  that  need  to  be  incorporated  into  the  simulation  model  and  what  can  be 
ignored  (p.  246).  He  further  explains  that  it  is  rarely  necessary  to  include  every  element 
of  the  system  in  the  model  in  order  to  make  effective  decisions,  and  in  some  cases  can 
cause  excessive  model  run  tune,  causing  missed  deadlines  and  obscuring  important 
factors  (Law,  2007). 
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Law  (2007)  presents  eight  general  guidelines  for  determining  the  level  of  detail 
required  when  constructing  a  simulation  model  (p.  247).  The  first  guideline  he  prescribes 
is  that  models  are  designed  for  a  specific  purpose  and  the  specific  issues  to  be 
investigated  and  their  corresponding  measures  ofperformance  must  be  defined  (Law, 
2007,  p.  247).  The  goal  of  the  research  is  defined  in  Chapter  1  and  must  be  thoroughly 
understood  prior  to  model  development.  Law  cites  an  example  in  which  “a  U.S.  military 
analyst  worked  on  a  simulation  model  for  six  months  without  interacting  with  the  general 
who  requested  it.  At  the  Pentagon  briefing  for  the  study,  the  general  walked  out  after  5 
minutes  stating,  That’s  not  the  problem  I’m  interested  in.”  (Law,  2007,  p.  247) 

Another  relevant  guideline  he  suggests  is  to  “use  subject-matter  experts  (SMEs) 
and  sensitivity  analyses  to  help  determine  the  level  of  model  detail”  and  scope  your 
model  to  focus  on  the  most  important  factors  (Law,  2007,  p.  247).  Specifically  important 
to  this  research  is  his  statement  under  this  guideline  that,  “Given  a  limited  amount  of  time 
for  model  development,  one  should  obviously  concentrate  on  the  most  important  lactors.” 
(Law,  2007,  p.  247)  The  consideration  on  the  amount  of  tune  needed  for  model 
development  relates  to  his  guideline  that  most  simulation  studies  have  a  time  and  money 
constraint  that  determines  the  level  of  detail  attained  in  the  model  (Law,  2007). 

In  the  final  two  guidelines  Law  directs  the  developer  away  from  including  more 
detail  than  is  necessary  to  address  the  issues  of  interest  while  keeping  enough  to  attain 
credibility  and  to  ensure  the  model  detail  is  consistent  with  the  type  of  data  that  is 
available  (Law,  2007).  The  summary  for  the  guidelines  prescribed  by  Law  is  to  clearly 
define  the  issues  of  interest,  to  have  enough  detail  in  the  model  to  consider  all  the  major 
lactors  that  influence  the  issue  of  interest  while  not  including  unnecessary  levels  of  detail, 
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and  to  consider  the  time  constraints  of  the  study  when  deciding  on  the  level  of  detail 
within  the  model 

Measures  of  Success 

Sadowskiand  Grabau  define  success  in  a  simulation  project  as  “one  that  delivers 
useful  information  at  the  appropriate  time  to  support  a  meaningful  decision”  (2004,  p. 

61).  They  give  three  elements  of  success  including  the  right  timing,  the  right 
information,  and  the  right  decision  (Sadowski  &  Grabau,  2004).  The  right  information 
includes  providing  information  that  the  study  stakeholders  are  interested  in  as  well  as 
insights  garnered  by  the  simulation  developer  (Sadowski  &  Grabau,  2004,  p.  6 1 ).  The 
right  timing  constitutes  providing  meaningful  information  to  decision  makers  in  time  to 
assist  their  decision  (Sadowski  &  Grabau,  2004,  p.  62). 

The  third  element  is  making  the  right  decision.  This  may  be  out  of  the 
researcher’s  control  but  in  order  for  the  results  to  be  of  value  they  must  be  delivered  to 
the  right  person  in  the  right  context  (Sadowski  &  Grabau,  2004).  If  the  study  results  in 
decision  level  data  but  never  makes  it  to  the  person  making  decision  then  the  study  may 
have  been  for  naught.  Sadowski  and  Grabau  (2004)  go  onto  discuss  the  potential  pitfalls 
and  problems  that  keep  simulation  developers  from  attaining  success  and  describe  ways 
to  avoid  those  pitfalls.  One  of  the  elements  of  success  was  influencing  the  right  decision 
and  in  order  for  that  to  happen  the  decision  maker  (manager)  needs  to  believe  and  use  the 
results. 

Little  (2004)  states,  “The  big  problem  with  management  science  models  is  that 
managers  practically  never  use  them”  (p.  1841).  He  give  three  reasons  for  what  is  wrong 
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including:  1)  good  models  are  hard  to  find  2)  good  parameterization  is  even  harder  and 
3)  managers  do  not  understand  the  models  (Little,  2004,  p.  1841).  As  the  manager  digs 
into  the  study  they  find  questionable  assumptions,  the  study  ignores  qualitative  aspects 
important  to  the  manager  and  uses  confusing  terminology  (p.  1841).  Additionally, 
modelers  tend  to  thrive  on  adding  more  and  more  detail  while  at  the  same  time 
constructing  an  incomplete  model  due  to  omitting  critical  phenomenon  (Little,  2004). 
Little’s  direction  is  to  strive  to  construct  a  simple  model  that  includes  quite  a  few 
phenomenon  (2004). 

In  order  to  assist  in  getting  managers  buy-in  on  model  use,  Little  (2004)  provides 
a  “decision  calculus”  or  a  set  of  procedures  to  ensure  the  model  is  used  to  assist  the 
manager  in  decisions  (p.  1843).  Little’s  (2004)  “decision  calculus”  includes  being 
simple,  robust,  easy  to  control,  adaptive,  complete  on  important  issues,  and  easy  to 
communicate  with  (p.  1843-1844).  Summarizing  his  methods,  they  are  designed  to 
include  the  manager’s  view  of  the  system  and  “make  the  model  more  a  part  of  him” 
(Little,  2004,  p.  1844).  Getting  the  manager  to  have  buy-in  on  the  simulation  model 
developed,  understand  the  model,  and  trust  the  results  will  be  at  the  forefront  of  this 
research.  Proper  verification  and  validation  is  one  way  to  gain  trust  in  the  model. 

Verification  &  Validation 

The  reasons  for  using  simulation  can  be  negated  if  the  model  does  not  reflect  real 
system  behaviors  or  if  the  outputs  do  not  provide  useful  insights  into  system  behaviors. 
Deliberate  verification  and  validation  (V  &  V)  ensures  the  model  is  designed  to 
accurately  depict  the  real  system  and  its  associated  behaviors.  According  to  Carson 
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(2005),  “The  results  of  the  V  &  V  phase  is  a  verified,  validated  model  that  is  judged  to  be 
accurate  enough  for  experimentation  purposes  over  the  range  of  system  designs 
contemplated”  (p.  21).  The  first  step  in  the  V  &  V  process  is  model  verification  (Banks, 
et  aL,  2010)  (Carson  II,  2005)  (Law,  2007). 

Verification 

According  to  Banks  et  aL  (2010)  stated,  “The  purpose  of  model  verification  is  to 
assure  that  the  conceptual  model  is  reflected  accurately  in  the  operational  model”  (p. 

390).  They  assert  a  few  common-sense  suggestions  for  the  verification  process  including 
having  the  operational  model  checked  by  someone  other  than  its  developer,  examining 
the  model  for  reasonable  outputs  under  a  variety  of  input  values,  verifying  that  what  is 
seen  in  an  animated  model  depicts  the  real  system  (Banks  et  al.,  2010,  p.  390-391). 

Banks  et  al.  (2010)  go  onto  discuss  a  common  failing  among  students  who  are  learning 
simulation,  identifying  that  the  most  often  overlooked  suggestion  “is  a  close  and 
thorough  examination  of  model  output  for  reasonable  ness  ”(p.  391).  They  also  suggest 
using  a  debugger  to  monitor  the  simulation  as  it  progresses  in  order  to  focus  on  certain 
areas  or  events  within  the  model  (Banks  et  aL,  2010,  p.  391).  An  important  part  of  the 
debugger  suggestion,  in  regards  to  this  research,  is  their  suggestion  to  monitor  the  values 
or  statuses  of  selected  components,  variables,  attributes,  etc.  (Banks  et  al.,  2010,  p.  391) 
Monitoring  the  values  of  resources  and  attributes  will  be  critical  during  model 
development  to  ensure  the  functions  not  visible  through  animation  are  operating  properly. 

Law  (2007)  discusses  eight  techniques  that  can  be  used  during  verification  of  a 
simulation.  His  first  technique  is  to  “write  and  debug  the  computer  program  in  modules 
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or  subprograms”  (Law,  2007,  p.  248).  Building  the  simulation  in  a  piecewise  fashion  and 
conducting  verification  prior  to  duplicating  the  functions  will  be  essential  to  minimizing 
errors  and  catching  them  early  in  development.  Law’s  second  technique  recommends 
using  a  group  of  experts  to  review  the  model  by  conducting  a  “structured  walk-through  of 
the  program”  (2007,  p.  249).  This  technique  allows  for  multiple  simulation  experts  to 
walk  through  a  simulation  and  verily  that  each  function  is  correct.  Discussions  with 
operations  researchers  and  professors  knowledgeable  in  simulation  will  be  valuable  to 
gaining  further  insights  into  the  art  of  simulation  but  also  to  catching  errors  within  the 
model  developed  for  this  research. 

In  his  third  technique  Law  recommends  running  the  simulation  under  a  variety  of 
input  parameter  settings  to  ensure  outputs  remain  logical  (2007,  p.  249).  This  technique 
foils  in  line  with  Banks  et  al.  (2010)  recommendations  to  continually  verily  and  validate 
the  model  by  frequently  comparing  model  outputs  to  the  real  system.  His  fourth 
technique  involves  using  a  “trace”  in  which  the  variable  or  statistical  values  are  displayed 
after  each  event  and  compared  to  real  data  to  verify  the  model  is  operating  correctly 
(Law,  2007).  The  final  techniques  Law  (2007)  recommends  are  to  run  the  model  with 
simplified  assumptions  to  compare  outputs  to  real  system  values  that  can  be  calculated, 
use  animation  to  observe  the  simulation,  compare  statistical  values  generated  from  the 
model  with  historical  data,  and  to  use  commercial  software  packages  to  reduce  the 
programming  required  (pp.  251-253).  Verification  produces  an  operational  model  that 
should  reflect  the  real  system  but  does  not  guarantee  accuracy.  Model  validation  gives 
the  outputs  of  the  model  accuracy  and  increases  model  c  red  foility. 
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Validation 


Carson  (2005),  Law  (2007),  Banks  et  aL  (2010),  and  Little  (2004)  emphasize  the 
importance  of  involving  managers  and  their  teams  in  validating  the  model.  Carson 
(2005)  states,  “Model  validation  gets  the  customer  involved”  (p.  21).  Banks  et  aL  (2010) 
defines  validation  as  “the  overall  process  of  comparing  the  model  and  its  behavior”  and 
calibration  as  ‘the  iterative  process  of  comparing  the  model  to  the  real  system,  making 
adjustments  (or  even  changes)  to  the  modeb  comparing  the  revised  model  to  reality, 
making  additional  adjustments,  comparing  again,  and  so  on”  (p.  395).  An  important 
point  to  highlight  is  the  recommendation  by  Banks  et  aL  (2010)  “for  the  modeler  to  use 
the  main  responses  of  interest  as  the  primary  criteria  for  validating  the  model”  (p.  398). 
The  primary  responses  of  interest  should  be  validated  against  actual  data  since  the  F-22 
Heavy  Maintenance  Modification  program  is  an  existing  system  with  numerous  data 
points  for  comparison. 

Banks  et  aL  (2010),  Law  (2007),  and  numerous  other  simulation  practitioners 
highlight  the  benefits  of  continually  verifying  and  validating  the  model  during 
development.  The  importance  of  creating  a  model  that  not  only  reflects  the  current 
system  but  also  produces  logical  outputs  as  input  parameters  change  cannot  be 
overstated.  Carson  (2002)  quotes  the  statistician  George  Box  in  saying  “All  models  are 
wrong.  Some  are  useful.”  (p.  53)  Proper  verification  and  validation  will  ensure  the 
simulation  outputs  provide  system  behavioral  insights  that  can  be  “useful”  for  managerial 
decisions. 
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Design  of  Experiments 


In  order  to  understand  the  cause- and -effect  relationships  in  a  system  the  input 
variables  must  be  deliberately  changed  while  observing  the  associated  response  changes 
of  the  output  variables  (Montgomery,  2013).  According  to  Montgomery  (2013), 
experimentation  is  a  criticalpart  of  the  scientific  method  and  well-defined  experiments 
can  lead  to  an  empirical  model  of  the  system  (p.  3). 

He  also  discusses  statistical  process  design  of  experiments,  referring  to  “the 
process  of  planning  the  experiment  so  that  appropriate  data  will  be  collected  and 
analyzed  by  statistical  methods”  (Montgomery,  2013,  p.  1 1)  and  that  research  needs 
deliberate  data  collection  and  statistical  analysis  for  adequate  insights  and  results  to  be 
realized.  Montgomery  (2013)  also  provides  severalbroad  reasons  for  running 
experiments,  one  of  which  is  discovery  (p.  2013).  Discovery  is  useful  for  trying  to 
explore  new  lactors  or  ranges  of  factors  (Montgomery,  2013),  similar  to  the  focus  ofthis 
research  on  discovering  the  new  factors  created  by  multi-skilling  specialties. 

Sanchez  (2005)  states,  “A  well-designed  experiment  allows  the  analyst  to 
examine  many  lactors  than  would  otherwise  be  possible,  while  providing  insights  that 
could  not  be  gleaned  fromtrial-and-error  approaches  or  by  sampling  lactors  one  at  a 
time”  (p.  69).  Her  assertions  focus  on  the  key  point  that  designing  an  experiment  well 
and  using  the  proper  methods  can  cut  down  on  sampling  requirements  significantly 
(Sanchez,  2005).  Montgomery  (2013)  and  Sanchez  (2005)  both  discuss  the  pitfalls  of 
adjusting  one  lactor  at  a  time  (OFA)  due  to  the  loss  of  insights  into  the  interaction  effects 
associated  with  the  varying  levels.  Both  parties  discuss  the  benefits  of  using  factorial 
designs,  fractional  designs,  and  Latin  hypercube  designs. 
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The  Latin  hypercube  design  is  one  of  interest  because  of  its  flexibility  to 
efficiently  design  experiments  while  retaining  space-filling  properties  of  full  factorial 
designs.  The  largest  benefit  of  a  Latin  hypercube  design  is  the  “number  of  designs  grows 
linearly  withk  rather  than  exponentially”  (Sanchez,  2005,  p.  76).  The  k  refers  to  the 
number  of  factors  considered  in  the  experiment.  The  linear  growth  significantly  reduces 
the  number  of  design  points  needed  to  get  accurate  insights  into  the  interactions  across 
the  factors  at  many  levels.  The  downside  of  a  randomized  Latin  Hypercube  is  that 
smaller  designs  may  have  pairwise  correlations  (Sanchez,  2005).  She  provides  a  remedy 
to  the  smaller  design  by  introducing  work  on  nearly  orthogonal  Latin  hypercube  (NOLH) 
designs  by  Cioppa  and  Lucas  (2005)  (Sanchez,  2005). 

Cioppa  and  Lucas  (2007)  explore  the  benefits  of  using  a  space- filling  design  of 
experiments  to  gather  insights  on  uncertaint  response  surfaces  and  their  application  to 
DoD  simulation  modeling.  Their  research  minimizes  the  unsampled  regions  of  an 
experimental  region  while  improving  the  space-filling  properties  of  orthogonal  Latin 
hypercubes  for  k  <  67  (Cioppa  &  Lucas,  2007).  The  ability  to  deal  with  a  smaller 
number  of  factors  is  desireable  for  potential  experimentation  options  in  this  research. 
Additionally,  their  research  provides  “readily  available  designs  that  allow  analysts  to 
explore  how  well  a  diverse  set  of  meta-models  captures  the  relationships  between  many 
input  variables  of  a  simulation  and  one  or  more  output  variables”  (Cioppa  &  Lucas,  2007, 
p.  54).  The  NOLH  design  they  provide  assists  in  examination  by  allowing  fitting  of 
models  with  main,  quadratic,  and  interaction  affects  with  nearly  uncorrelated  estimates  of 
regression  coefficients  for  the  linear  term  effects  (Cioppa  &  Lucas,  2007,  p.  54).  The 
spreasheet  they  implemented  is  available  at  http://harvest.nps.edu.  Summarized,  the 
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NOLH  design  allows  the  researcher  to  gain  valuable  insights  on  an  unknown  response 
surface  without  having  to  use  full  factorial  or  similar  designs,  which  require 
exponentially  increasing  design  points  as  factors  ( k )  increase. 

Marginal  Analysis 

The  objective  of  this  research  is  to  select  the  best  choice  ofpersonnel  policies  in 
regards  to  multi-skilling  versus  the  status  quo  of  overtime.  Marginal  analysis  is  a  basic 
form  of  optimization  that  assists  in  selecting  from  among  differing  technically  efficient 
alternatives  (de  Neufville,  1990,  p.  41).  According  to  de  Neufville  (1990),  “marginal 
analysis  combines  the  production  function,  which  represents  only  the  technically  efficient 
production  possibilities;  and  the  input  cost  function,  which  describes  the  cost  of  inputs 
used”  (1990,  p.  41). 

One  of  the  key  assumptions  relating  to  this  research  is  that  the  only  constraint  on 
resources  is  the  money  available  or  the  budget  (de  Neufville,  1990).  Marginal  analysis  is 
useful  for  optimizing  the  system  design  through  simulation  if  the  costs  of  inputs  are 
readily  available.  The  cost  data  for  this  research  is  available  from  the  business  case 
analysis  completed  by  Ogden  ALC  onthe  proposed  Multi- Trade  Demonstration  Project 
discussed  earlier  in  this  chapter  (Ogden  Air  Logistics  Center,  2012).  For  this  study  the 
production  function  represents  model  output  changes  associated  with  resource 
adjustments  and  will  provide  insights  into  the  increases  or  decreases  in  system 
performance  as  well  as  the  cost  to  get  that  performance. 

Ysebaert  (2011)  used  marginal  analysis  by  implementing  a  “shopping  list” 
approach  to  gauge  the  improvements  of  Adjusted  Stock  Level  panels  for  the  F-22.  Her 
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marginal  analysis  measured  the  improvements  to  availability  compared  to  the  cost  of  an 
additional  panel  (Ysebaert,  2011).  Ysebaert’s  results  provided  valuable  insights  into  the 
aircraft  availability  hnprovements  possible  with  purchases  ofadditionalpanels  as  well  as 
the  cost  of  those  improvements  (2011 ).  Her  final  analysis  provided  the  “shopping  list”  or 
the  “what  to  purchase  next”  for  panels  included  as  well  as  the  improvement  costs 
associated  with  each  additional  panel  purchased.  Similar  methods  are  used  in  the  Aircraft 
Sustainability  Model  (ASM)  by  comparing  optimal  spares  mixes  based  on  a  target  metric 
defined  by  the  user  (Ysebaert,  2011)  (Slay,  Bachman,  Kline,  O’Malley,  Eichorn,  &  King, 
1996).  Both  Ysebaert  (2011)  and  Slay  et  al.  (1996)  use  marginal  analysis  based  on 
simulation  outputs  to  decide  the  “benefit  per  dollar”  and  the  scenarios  with  the  highest 
benefit  should  be  at  the  top  of  the  “shopping  list”.  Understanding  the  benefit  of  marginal 
analysis  leads  to  the  review  of  literature  on  selecting  the  best  experimental  scenario  in 
order  to  gain  the  best  value  for  the  taxpayers  dollar. 

Selecting  the  Best  Alternative 

Comparing  the  different  system  policies  of  multi-skilling  is  the  primary  reason  for 
selecting  simulation  as  a  methodology  in  this  research.  For  our  system,  the  goal  is  not 
just  to  compare  alternatives  but  also  to  select  the  best  of  the  differing  alternatives  through 
statistically  proven  ranking-and-se lection  (R  &  S)  methods. 

Kim  and  Nelson  (2006)  give  four  comparison  problems  that  arise  in  simulation 
studies  including  comparing  alternatives  against  a  standard,  selecting  the  best  performing 
system,  selecting  the  system  with  the  highest  probability  of  performing  the  best,  and 
selecting  the  largest  probability  of  success  (p.  501).  Their  emphasis  is  on  the  constraint 
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applied  to  probability  of  the  correct  selection  (PCS)  (Kim  &  Nelson,  2006).  Kim  and 
Nelson  (2006)  attribute  the  foundation  of  R  &  S  to  two  papers,  “Bechhofer  (1954) 
established  the  indifference-zone  formulation,  while  Gupta  (1956,  1965)  is  credited  with 
the  subset  selection  formulation  of  the  problem”  (p.  503).  They  espouse,  “Both 
approaches  were  developed  to  compensate  for  the  limited  inference  provided  by 
hypothesis  tests  for  the  homogeneity  ofk  population  parameters  (usually  means)”  (Kim 
&  Nelson,  2006,  p.  503). 

The  subset  selection  formulation  of  Gupta  has  a  goal  of  obtaining  a  subset  /  Q  {1, 
2,  ...,K}  suchthat  the  probability  ofselecting  the  best/  is  1 -a  (Kim  &  Nelson,  2006,  p. 
503).  a  refers  to  the  probability  of  not  selecting  the  best  alternative  or  eliminating  a 
system  when  in  fact  it  is  the  best.  /  is  the  retained  system  that  is  the  best  or  if  multiple 
systems  are  retained  they  are  better  than  the  others  but  are  not  significantly  different  from 
each  other.  It  does  this  by  selecting  the  system  with  the  mean  that  is  significantly  better 
than  the  other  output  means.  The  disadvantage  of  this  method  is  the  retained  set  /  may 
include  several  systems.  The  Indifference- zone  formulation  assists  in  alleviating  the 
disadvantage  of  keeping  multiple  systems  in  set /by  guaranteeing  to  select  the  single  best 
system  when  the  difference  in  the  means  of  the  two  systems  is  greater  than  a  certain  level 
of  indifference.  According  to  Kim  and  Nelson  (2006)  “The  indifference  level  is  the 
smallest  difference  the  experimenter  feels  is  worth  detecting.”  (p.  504)  Additionally, 
they  refer  to  Hsu  (1996,  pp.  100- 102)  and  his  conclusions  that  R  &  S  links  to  multiple 
comparison  procedures  (MCPs)  by  complete  multiple  comparisons  of  the  best  (MCB). 
Kim  and  Nelson’s  (2006)  analysis  concludes  that  “Bechhofer’s  and  Gupta’s  procedures 
can  be  augmented  with  MCB  confidence  intervals  “for  free”,  and  Bechhofer’s  procedure 


22 


is  guaranteed  to  select  a  system  within  the  5  of  the  best”  (Kim  &  Nelson,  2006,  p.  508). 
Note  that  6  (delta)  is  the  level  of  indifference  decided  upon  by  the  experimenter.  Refer  to 
Kim  and  Nelson  (2006)  for  discussion  and  detailed  steps  on  selections  techniques  as  well 
as  other  multiple  comparison  methods. 

Nelson,  Swann,  Goldsman,  and  Song  (2001)  combine  the  screening  procedures  of 
subset  selection  and  the  indifference  zone  (IZ)  by  introducing  a  decomposition  lemma. 
The  decomposition  lemma  “establishes  that  under  very  general  conditions  we  can  apply 
anIZ  selection  procedure  to  the  survivors  of  a  screening  procedure  and  still  gaurantee  an 
overall  probability  of  correct  selection  (CS)  even  if  the  selection  procedure  starts  with  the 
same  data  used  for  screening”  (Nelson  et  al.,  2001,  pp.  952-953).  The  decomposition 
lemma  allows  the  IZ  selection  procedure  to  be  applied  to  the  systems  retained  by  the 
screening  procedure  in  order  to  select  the  best  system  if  the  systems  differ  by  5  (Nelson  et 
al,  2001,  p.  953).  Nelson  et  al.  (2001)  then  offer  combined  procedures  for  selecting  the 
best  system  and  experimental  results  that  validate  its  ability  to  find  the  best  system  with  a 
confidence  level  of  at  least  1-a. 

Banks  et  al  (2010)  provide  a  six  step  procedure  for  Select- the- Best  procedure  (p. 
478)  to  apply  Nelson  et  al’s  (2001)  methodology. 

1.  Specify  the  desired  probability  of  correct  selection  1/k  <  l-a<l,the  practically 
significant  difference  £  >  0,  an  initial  number  of  replications  Ro  >  10,  and  the 
number  of  competing  systems  K.  Set 

t  =  t  1  (1) 

l-<l-a/2)— Jto-l 

and  obtain  Rinott’s  constant  h-  h(Ro  K,  l-a/2). 
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2.  Make  Ro  replications  of  each  system  and  calculate  the  first-stage  sample  means 
and  sample  variances. 

s2+s2 

3.  Calculate  the  screening  thresholds  WiJ =t(— - -)12  (2) 

Ro 

for  all  i£j. 

b)  If  smaller  is  better,  then  form  the  survivor  subset  S  containing  every  system 
design  I  such  that  Yi<Yj+  max  jo,  Wfj  -  e}  (3) 

for  all  j  ^  i. 

4.  If  the  set  S  contains  only  one  system,  then  stop  and  return  the  system  as  best.  If 

not,  compute  the  second-stage  sample  sizes  Rj  =  max  j R() , \(hSi /  e)2_ Ij  (4) 

where  p-|  means  to  round  up. 

5.  Take  additional  replications  R;  -  Ro  from  all  systems  i  in  S,  or  if  it  is  more 
convenient  obtain  a  total  of  R  replications  by  starting  over. 

6.  Compute  the  overall  sample  means  for  all  i  in  S.  If  smaller  is  better,  select  the 
system  with  the  smallest  mean.  (Banks  et  aL,  2010,  p.  479-480) 

Banks  et  aL  (2010)  provide  an  automated  tool  that  implements  these  procedures  in 
the  SimulationTools.xls  spreadsheet  at  www.bcnn.net  (p.  481).  Banks  et  al.  (2010)  also 
note  the  research  by  Nelson  et  al  (2001)  and  the  ability  of  Select- the-Best  Procedure  to 
select  the  best  system  with  a  confidence  of  1-a  (p.  480).  Understanding  how  simulation 
can  be  useful  as  well  as  the  methods  for  selecting  the  best  performing  system  is  useful  as 
the  researcher  reviews  previous  simulation  efforts. 
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Works  in  Depot  Maintenance,  Aircraft  Maintenance,  and  Multi-skilling  Simulation 


A  plethora  of  simulation  research  is  readily  available  for  review  and  analysis.  A 
small  segment  of  the  research  involves  Department  of  Defense  (DoD)  related  topics  on 
depot  maintenance  activities  with  more  existing  in  aircraft  availability  as  well  as  cross¬ 
training  and  optimizing  workforces.  The  researcher  provides  insights  from  a  few  of  the 
many  reviewed  in  these  areas. 

As  discussed  previously,  Ysebaert  (2011)  conducted  a  simulation  study  on  aircraft 
availability  improvements  associated  with  using  and  adding  additional  Adjusted  Stock 
Levelpanels.  The  Adjusted  Stock  Level  panels  are  extra  LO  panels  held  in  supply  and 
changed  out  for  a  damaged  panel,  returning  the  aircraft  to  fully  mission  capable  status 
while  the  damaged  panel  is  repaired  (Ysebaert,  2011).  Her  simulation  was  developed  in 
ARENA  in  a  stepwise  lashionto  facilitate  verification  during  development. 

The  input  data  used  included  theoretical  distributions  based  on  historical  and 
aircraft  technical  data.  One  problematic  area  for  Ysebaert  is  that  the  data  pulled  from 
Production  System  Effectiveness  Data  System  was  inaccurate  for  a  panel  repair  time  and 
frequency  of  repair.  SMEs  inputs  were  used  in  order  to  estimate  repair  tunes  (constant) 
and  repair  frequency. 

Ysebaert’s  (2011)  results  include  using  marginal  analysis  to  develop  a  “shopping 
list”  of  pane  Is  based  on  the  order  panel  selection  that  provided  the  most  improvements. 
Using  Paired-t  tests  she  concludes  that  no  statistically  significant  difference  in 
availability  exists  after  purchasing  the  first  panel  but  practical  significance  exists 
(Ysebaert,  2011).  Additionally,  her  “shopping  list”  changed  when  she  used  the 
availability  to  cost  ratios.  Ysebaert  (2011)  concluded  that  not  every  panel  offers  the  same 
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return  for  increased  availability  or  decreased  panel  hours  and  that  there  is  a  limit  to  the 
benefit  from  increasing  Adjusted  Stock  Level  Panels  past  five  and  six  panels  (p.  61). 

Levien’s  (2010)  research  included  a  simulationof  the  KC-135  depot  maintenance 
IDOCK  process,  focusing  specifically  on  whether  multi-skilling  improved  the  BA  major 
job.  Further  scoping  the  research,  he  only  considered  three  of  the  six  maintenance 
specialties  used  in  the  BA  tasks  (Levien,  2010).  His  model  was  built  with  parallel  and 
subsequent  processes  at  the  task  level  detail  in  order  to  measure  changes  in  the  number  of 
aircraft  throughput,  employee  utilization  changes,  IDOCK  space  utilization,  and  each 
tasks  waiting  tune  (Levien,  2010). 

He  noted  data  problems  similar  to  Ysebaert  (2011)  associated  with  the  data  from 
previously  completed  maintenance  actions  (Levien,  2010).  Additionally,  validation 
versus  the  real  system  proved  quite  difficult  with  a  difference  of  over  22.7  days  (72.5 
percent)  for  a  process  that  takes  31.3  days  (Levien,  2010).  Levien  (2010)  concluded  that 
although  validation  proved  difficult,  the  relative  improvement  in  the  model  provides 
valuable  insights  on  cross-training  (multi-skilling)  personnel. 

Levien’s  (2010)  results  showed  that  major  job  BA  would  have  13.7  percent 
improvement  in  completion  tune  by  cross-training  employees  on  22  of  the  58  tasks  with 
lower  wait  time  for  four  of  the  six  major  operations  (p.  40).  The  results  of  his  research 
can  be  applied  to  study  the  entire  F-22  depot  maintenance  process  to  garner  similar 
insights. 

The  final  research  discussed  is  Park  (1991)  research  on  cross-training  in  a  dual 
resource  constrained  shop.  The  reason  for  discussing  an  older  study  is  due  to  the 
relevancy  to  the  multi-skilling  topic  in  this  research  and  Park’s  focus  specifically  on  the 
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personnel  resource  without  consideration  for  machine  limits.  Park’s  (1991 )  research 
focused  on  dual  resource  constrained  systems,  where  both  personnel  and  resources  were 
needed  to  complete  a  job  but  excess  machines  exist.  He  used  a  SLAM  simulation  to 
model  five  work  centers  at  varying  levels  of  cross-trained  personnel  and  with  differing 
work  dispatch  policies  (Park,  1991). 

Park  (1991)  concludes,  “The  minimum  introduction  of  worker  cross-training 
showed  the  most  significant  improvement,  and  subsequent  increase  in  cross-training  had 
diminishing  return”  (p.  298).  His  point  that  training  workers  in  at  least  two  work  centers 
should  be  the  focus  over  training  them  in  all  work  centers  and  that  cross-training  efforts 
may  need  to  be  focused  in  bottleneck  work  centers  (Park,  1991)  supports  the  business 
case  analysis  completed  by  Ogden  ALC  on  the  Multi- Trade  Demonstration  Project 
(Ogden  Air  Logistics  Center,  2012). 

The  literature  reviewed  in  this  section  includes  a  background  on  multi-skilling 
(cross-training),  ways  to  develop  and  use  a  simulation,  methods  for  analysis  of  outputs, 
and  a  review  of  relevant  simulation  studies.  During  the  process,  a  gap  in  research 
presents  itself  in  the  area  of  workforce  simulation  within  Air  Force  depot  maintenance 
activities.  The  knowledge  gained  during  this  process  is  the  starting  point  for  this 
research. 
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ITT.  Methodology 


Chapter  Ove  rvie  w 

The  first  portion  of  this  chapter  describes  the  process  of  the  F-22  Heavy 
Maintenance  Modification  Program  (depot  line)  and  the  methods  used  for  development 
and  validation  of  the  discrete  event  simulation  in  ARENA  14®.  The  second  portion  of 
this  chapter  discusses  the  experimental  methods  used  for  comparing  civilian  specialties 
for  multi-skilling. 

The  course  ofthe  background  and  methods  used  in  my  research  were  honed  using 
574th  Aircraft  Maintenance  Squadron  (AMXS),  F-22  Flight  subject  matter  expert  (SME) 
opinions,  knowledge  and  insights  into  the  maintenance  personnel,  resources  and  tasks 
that  characterize  the  F-22  Heavy  Maintenance  Modification  Program 

F-22  Heavy  Maintenance  Modification  Program 

The  F-22  depot  maintenance  operation  (system)  is  constructed  similarly  to  other 
ALC  depot  maintenance  activities  within  the  Air  Force.  The  F-22  is  unique  in  that  each 
aircraft  undergoes  specified  maintenance  actions  based  on  the  modifications  required  for 
each  production  year.  Additional  maintenance  actions  completed  include  Tune 
Compliance  Technical  Orders  (TCTOs),  delayed  maintenance  discrepancies  that 
operational  units  request  completed,  and  over  and  above  (O&A)  actions  that  maintenance 
personnel  find  during  the  depot  process. 

The  object  of  primary  interest  (entity  or  flow  unit)  that  enters  and  exits  the  system 
is  the  F-22.  All  discuss  ion  regarding  the  system  boundaries,  processes,  maintenance 
actions,  etc  will  be  predicated  on  the  understanding  that  an  aircraft  flowing  through  the 
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F-22  depot  maintenance  operation  is  the  primary  focus  of  this  research.  The  process  or 
system  boundaries  for  this  research  need  to  be  defined  prior  to  delving  into  specific 
characteristics  of  the  system. 

The  system  boundaries  are  defined  based  on  the  interdependence  of  maintenance 
resources  and  activities  towards  the  completion  of  depot  maintenance  on  each  F-22 
(Banks  et  aL,  p.  12).  More  narrowly  defined,  the  system  is  identified  by  the  point  at 
which  inputs  begin  to  be  transformed  into  outputs  (Anupindi,  Chopra,  Deshmukh  D,  Van 
Mieghem  A,  &  Zemel.  2011).  We  defined  inputs  as  any  F-22  that  arrives  for  scheduled 
induction  into  the  F-22  depot  line  at  Ogden  ALC.  Outputs  from  the  system  are  defined  as 
an  aircraft  with  all  scheduled  and  O&A  maintenance  actions  complete  that  is  ready  for 
return  to  the  owning  organization.  Based  on  the  these  definitions  of  what  constitutes  a 
system  or  process,  we  define  the  system  boundaries  for  this  research  as  starting  when 
F-22  depot  personnel  begin  scheduled  maintenance  actions  on  an  aircraft  and  as  stopping 
when  all  maintenance  actions  are  complete.  The  clearly  delineated  system  boundaries 
allow  for  a  further  discussion  of  the  flow  of  an  aircraft  through  the  depot  process. 

Understanding  the  flow  of  an  aircraft  within  the  system  requires  a  breakdown  of 
the  process  and  resources  within  the  system.  Each  F-22  arrives  at  scheduled  intervals 
determined  by  engineering  specifications  and  based  on  collaborative  planning  between 
the  F-22  Flight  and  organizations  that  own  the  aircraft.  Each  aircraft  arrives  with  a 
preplanned  number  of  maintenance  actions  scheduled  based  on  modifications  required, 
the  delayed  discrepancies  identified  for  completion  and  any  other  O&A  actions  found 
during  the  depot  process.  The  aircraft  is  scheduled  to  be  in  the  system  for  a 
predetermined  number  of  workdays  based  on  maintenance  man-hour  requirements  with 
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all  maintenance  actions  scheduled  for  completion  within  this  time  frame.  The  number  of 
flow  days  schedule  for  each  aircraft  is  broken  down  into  seven  measureable  segments 
referred  to  as  gates.  The  seven  gates  exist  as  milestones  within  the  process  to  focus 
maintenance  efforts  on  meeting  predetermined  timelines  and  goals  for  process 
completion.  Management  and  production  supervisors  restrict  personnel  to  working  only 
maintenance  actions  required  within  the  gate  the  aircraft  is  currently  assigned  and 
maintenance  actions  within  the  next  gate  do  not  begin  until  all  actions  from  the  previous 
gate  are  complete  with  rare  exceptions.  F  igure  2  shows  an  example  of  the  gates  by  day. 
Scheduling  maintenance  tasks  and  assigning  them  within  each  gate  requires  each 
maintenance  specialty  to  be  identified. 
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Figure  2:  Gate  Example 

There  are  six  Federal  Wage  Series  (FWS)  occupational  specialties  that  complete  a 
majority  of  the  maintenance  on  the  F-22  depot  line.  The  specialties  are  Painter  with 
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specialization  in  low  observable  (LO)  applications,  Aircraft  Mechanic,  Sheet  Metal 
Mechanic,  Aircraft  Fuels  Systems  Mechanic,  Aircraft  Electrician,  and  Avionics 
Technicians  (Ogden  Air  Logistics  Center,  2012).  A  critical  path  analysis  by  the  574th 
Aircraft  Maintenance  Squadron  (AMXS),  F-22  Flight  planners  identifies  maintenance 
activities  of  these  six  specialties  as  critical  path  tasks.  The  Critical  Path  analysis  focuses 
my  research  on  the  specialties  identified  because  improvements  off  the  critical  path  will 
not  improve  overall  system  performance.  F-22  Flight  planners  previously  enlisted  the 
help  ofMariahMagagnotti  and  Joseph Hasler  (both PhD  students  at  Clemson 
University),  and  Dr  Scott  J.  Mason  from  Clemson  University  to  produce  an  automated 
scheduling  tool  to  identify  man-hour  requirements  per  specialty  per  day.  F igure  52 : 
Clemson  Scheduling  Tool  (MORS)  Example,  in  Appendix  A,  shows  an  example  of 
several  days  of  the  man-hour  requirements  from  the  tool.  Analysis  of  this  tool  using 
actual  man-hours  consumed  on  multiple  aircraft  shows  that  the  six  specialties  identified 
previously  account  for  97  percent  of  the  total  man-hours  required  to  complete 
maintenance  actions  on  an  aircraft.  SME  input  and  analysis  of  the  Clemson  tool  confirms 
this  research  focus  on  the  Painter  (LO  or  AP),  Aircraft  Mechanic  (AG),  Sheet  Metal 
Mechanic  (AS),  Aircraft  Fuels  Systems  Mechanic  (AT),  Aircraft  Electrician  (AR),  and 
Aircraft  Avionics  (AV)  FWS  occupational  specialties. 

The  final  portions  of  the  system  that  must  be  identified  prior  to  model 
development  are  the  resources  required  for  maintenance  activities.  The  resources  include 
maintenance  personnel  by  specialty,  general  maintenance  docks,  and  LO  docks.  The  six 
FWS  occupational  specialties  will  be  the  primary  resource  of  focus  for  this  research. 
Maintenance  personnel  and  associated  activities  operate  24  hours  a  day,  Monday  to 
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Friday  with  every  other  Friday  off  and  differing  numbers  of  personnel  per  specialty 
scheduled  for  each  shift.  The  naming  convention  of  the  docks  explains  the  type  of 
maintenance  that  is  completed  in  each  one.  The  one  caveat  is  that  general  maintenance 
activities  can  be  completed  in  maintenance,  or  LO  docks  but  LO  maintenance  is  only 
comp Icted  in  LO  docks.  Additional  resources  are  required  during  the  depot  process  but 
SME  input  narrowed  the  constraining  resources  (outside  of  personnel)  down  to  these  two 
for  modeling  purposes. 

F-22  Depot  Operations’  workload  will  be  changing  significantly  over  the  next 
year.  Currently  the  number  of  aircraft  within  F-22  depot  maintenance  operations  at 
Ogden  ALC  is  restricted  to  a  specific  number  at  any  one  tune.  The  depot  maintenance 
work  is  currently  split  between  Ogden  ALC  and  the  Lockheed  facility  in  Palmdale,  CA 
but  the  Air  Force  recently  announced  the  consolidation  of  all  F-22  depot  maintenance 
activities  to  Ogden  ALC  (United  States  Air  Force,  2013).  The  new  workload  will  double 
the  number  of  aircraft  in  the  system  with  a  corresponding  increase  in  annual  throughput. 
Doubling  the  workload  and  associated  resources  of  the  depot  operation  combined  with 
the  current  overtime  and  utilization  problems  previously  discussed  makes  simulation 
research  on  multi-skilling  extremely  timely  and  relevant  for  decision  makers. 
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Model  Development 


Model  conceptualization  starts  by  identifying  the  goal  of  the  model.  Stephen 
Covey’s  Second  Habit  in  The  7  Habits  of  Highly  Effective  People  is  to  “Begin  with  the 
End  in  Mind”  (2004).  We  start  model  conceptualization  by  reverting  to  the  initial 
research  questions  to  clearly  understand  the  goal  of  this  research. 

Research  Question  1:  Which  maintenance  specialties  within  F-22  Depot  Operations 
should  be  considered  for  multi-skilling  into  the  low  observable  (LO)  specialty? 

Research  Question  2:  How  does  multi-skilling  affect  aircraft  throughput  and  employee 
utilization  in  F-22  Depot  Operations? 

Research  Question  3:  To  what  extent  does  multi-skilling  offset  overtime  use  to  meet 
aircraft  flow  day  requirements  and  throughput  targets? 

The  goal  of  our  model  is  to  give  a  quantitative  basis  for  multi-skilling  as  well  to 
gain  insight  into  specialty  selection  for  multi-skilling.  The  basis  required  for  our  decision 
analysis  is  whether  or  not  there  is  an  increase  in  aircraft  throughput  with  an  associated 
decrease  in  time  an  aircraft  spends  in  the  system  and  whether  or  not  improvements  in 
employee  utilization  are  realized  from  multi-skilling.  With  these  goals  in  mind,  the 
model  needs  to  be  able  to  capture  the  following  outputs: 

•  Throughput 

•  Average  flow  days 

•  Average  flow  days  by  type  of  aircraft  (based  on  hours) 

•  Time  in  each  gate 

•  Employee  man-hour  use  by  specialty 

•  Employee  use  in  secondary  skill  (multi-skill  use) 
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•  Employee  percentage  use  in  secondary  skill 

•  Overtime 

•  Dock  usage 

•  Dock  queue  time 

The  desired  outputs  are  key  considerations  in  model  development  to  ensure  the 
final  results  and  associated  recommendations  have  decisional  implications.  The  next  step 
is  to  develop  a  conceptual  model  using  the  process  insights  of  the  F-22  Heavy 
Maintenance  Modification  Program  and  desired  outputs  of  the  simulation. 

Scope  Decision 

The  decision  on  the  scope  of  the  simulation  and  the  level  of  detail  needed  for 
decision  making  must  be  addressed  prior  to  model  conceptualization.  According  to 
Sadowskiand  Grabau,  it  is  easy  to  fall  into  the  trap  of  “Getting  Lost  in  Detail”  (2004). 
The  decision  on  the  level  of  detail  required  in  order  to  perform  substantive  analysis  of  the 
system  is  critical  in  this  study. 

The  initial  direction  of  the  modeling  effort  is  to  scope  the  model  to  a  manageable 
level  of  detail  by  identifying  the  segments  of  the  real  system  to  be  included.  The  first 
course  considered  is  to  model  a  few  of  the  seven  gates  within  the  system  in  order  to 
alleviate  undue  levels  of  detail  associated  with  the  size  and  complexity  of  the  system. 
Modeling  two  or  three  gates  will  give  an  idea  of  output  behaviors  associated  with  those 
gates  as  multi-skilling  is  implemented  and  may  give  an  accurate  representation  of  the 
overall  system  behaviors  associated  with  multi-skilling. 
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However,  the  problem  with  including  only  certain  gates  is  that  man-hour 
requirements  per  specialty  vary  greatly  between  each  gate  and  isolating  the  model  to 
certain  gates  may  inaccurately  show  higher  manpower  availability  than  actually  exists 
within  the  entire  system  if  the  wrong  gates  are  included.  The  impact  of  the  inaccurate 
resource  availability  will  show  more  or  less  benefit  from  multi-skilling  than  really  exists, 
depending  on  the  resources  that  display  the  inaccurate  availability  and  the  degree  to 
which  the  availability  is  over  or  understated.  Further  complicating  the  problem  of 
identifying  gates  for  inclusion  is  that  SME  inputs  identify  periods  when  excess  resource 
(personnel)  capacity  exists  due  to  inactivity  in  a  gate  (i.e.  no  aircraft  in  the  gate). 
Management  and  planners  are  unable  to  forecast  the  gates  and  periods  when  this  excess 
capacity  will  arise,  leading  to  a  shift  to  model  all  seven  gates  versus  scoping  the  model  to 
a  few.  Insights  gained  from  reviewing  multi-skilling  research  by  Major  Andrew  Levien 
further  supports  the  shift  to  developing  a  model  that  captures  the  entire  system 

Levien’s  multi-skilling  simulation  study  on  the  KC- 135  depot  process  considers 
one  small  segment  of  the  overall  process  and  is  a  good  first  step  into  multi-skilling 
simulation  research  within  the  ALC’s  (2010).  However,  his  research  does  not  provide 
insights  from  a  system  perspective  that  can  be  used  for  managerial  decisions  on  multi¬ 
skilling  because  interactions  of  maintenance  tasks  and  personnel  within  depot  processes 
are  complex  and  isolated  process  improvements  (local  optimization)  may  not  impact 
overall  system  performance.  On  the  other  hand,  improvements  in  employee  utilization  in 
certain  areas  may  have  an  unforeseen  impact  on  another  process  within  the  system, 
influencing  overall  system  performance  measures.  The  model  needs  to  include  all  gates 
in  order  to  ensure  potential  impacts  are  not  hidden  due  narrowing  the  scope  of  the  model. 
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Additionally,  the  goal  of  this  research  is  to  provide  adequate  decision  level  insights  into 
the  impact  multi-skilling  will  have  onF-22  Heavy  Maintenance  Modification  Program 
performance  measures  and  modeling  the  entire  system  is  the  only  way  to  ensure  those 
insights  are  captured.  After  deciding  to  model  the  entire  system,  the  next  step  is  to  decide 
on  the  level  of  detail  to  include  within  the  model. 

The  optimal  level  of  detail  is  to  model  the  system  at  task  level  in  order  to  most 
accurately  capture  the  benefits  of  multi-skilling.  Task  level  detail  allows  for  movement 
of  personnel  between  tasks  at  shorter  periods,  permitting  the  model  to  measure  the  use  of 
multi-skilled  personnel  and  benefits  in  greater  detail  than  an  aggregate  level.  However, 
the  time  constraints  of  the  study,  the  sheer  number  of  maintenance  tasks  (three  to  four 
thousand)  required  on  each  aircraft,  and  the  variability  of  the  number  of  tasks  from  one 
aircraft  to  the  next  makes  modeling  all  tasks  impractical  for  this  study.  Data  accuracy 
concerns  also  exist  at  the  task  level. 

The  data  for  task  level  analysis  is  found  not  to  be  accurate  after  considering  SME 
inputs  and  current  work  practices.  Maintenance  technicians  complete  tasks  in  a  given 
area  on  the  aircraft  that  may  overlap  multiple  work  cards.  Many  tunes  the  man-hours 
used  to  complete  the  preparation  task  are  only  reflected  in  the  data  for  one  of  the  many 
tasks  that  required  the  prep  work.  For  example,  sanding  a  structural  area  of  the  aircraft 
may  be  required  if  you  remove  one  panel  or  all  four  in  the  area  but  the  man-hours  are 
usually  captured  in  only  one  of  the  task  times.  If  you  simulate  at  the  task  level  the  data 
will  reflect  longer  tunes  for  certain  tasks  and  shorter  times  for  others  with  no  consistency 
to  which  one  the  sanding  hours  are  applied.  The  data  discrepancy  would  not  be  a 
problem  if  all  the  panels  were  removed  every  time  but  each  aircraft  is  different  and 
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requires  differing  levels  of  maintenance.  An  additional  consideration  is  how  the 
inconsistency  of  the  man-hour  allocations  will  affect  input  distributions  for  the  task 
times.  The  data  discrepancies  described  and  time  limitations  of  the  study  steer  the 
research  towards  a  more  aggregate  level  simulation  of  the  system 

Deciding  on  the  level  of  aggregation  is  complicated  given  the  amount  of  data 
available  and  number  of  tasks  associated  with  each  aircraft.  Since  variability  of  man¬ 
hour  requirements  is  the  most  significant  challenge  that  multi-skilling  is  proposed  to 
address,  the  key  [actor  in  determining  the  level  of  aggregation  for  the  model  is  the  ability 
to  simulate  multiple  types  of  aircraft  from  a  man-hour  requirement  perspective. 

Two  questions  are  considered  in  the  aggregation  decision.  1)  What  data  is 
currently  available  and  2)  What  type  of  data  do  managers  and  planners  use  for  their 
scheduling/ forecasting?  Searching  to  answer  the  first  question  revealed  that  data  is 
available  from  the  task  level  up  to  the  total  man-hours  required  per  aircraft  along  with 
many  derivatives  of  the  data  in  between  the  two  levels.  Availability  of  data  is  not  the 
problem  and  understanding  the  types  of  data  gives  a  clear  picture  of  the  numerous 
modeling  options  but  does  not  assist  in  choosing  the  level  of  aggregation  for  the 
simulation. 

The  answer  to  the  second  question  leads  to  the  realization  that  the  F-22  planners 
currently  use  aggregate  data  output  fromthe  Clemsontool  to  forecast  manpower  needs, 
aircraft  flow  through  the  system,  and  to  produce  a  visual  layout  of  all  aircraft  schedules 
by  flow  day.  The  Clemson  toolproduces  the  aggregate  man-hour  requirements  per 
specialty  per  flow  day  for  an  aircraft.  The  tool  pulls  from  Programmed  Depot 
Maintenance  Scheduling  System  (PDMSS)  outputs  the  standard  hours  required  based  on 
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the  projected  maintenance  requirements  of  an  aircraft  or  pulls  in  the  actual  hours  used  to 
complete  an  aircraft.  Most  decisions  within  the  F-22  Heavy  Maintenance  Modification 
Program  are  viewed  from  the  man-hour  requirement  perspective  and  the  simulation 
model  should  incorporate  that  paradigm. 

Using  data  that  managers  and  their  team  members  are  femiliar  with  facilitates 
getting  their  buy- in  as  the  model  is  developed  and  also  allows  for  more  accurate  feedback 
from  SMEs  on  the  details  of  the  system  during  model  development.  Additionally,  using 
the  aggregate  data  mitigates  some  of  the  task  level  data  anomalies  because  the  tasks  are 
combined  and  include  all  tasks  performed  by  that  specialty  on  a  specified  day.  For  these 
reasons,  the  plan  is  to  build  the  model  with  the  intent  of  simulating  the  flow  of  an  aircraft 
through  the  system  with  maintenance  actions  on  each  aircraft  being  performed  based  on 
man-hour  requirements  per  specialty  per  day.  The  next  step  following  the  decision  on  the 
scope  of  the  model  and  the  level  of  detail  is  to  begin  conceptualization  of  the  model. 

Conceptualization 

The  first  portion  of  conceptual  development  of  the  model  is  to  layout  the  flow  of 
the  F-22  Heavy  Maintenance  Modification  Program  SMEs  are  consulted  frequently 
during  each  stage  of  development  in  order  to  accurately  capture  details  of  the  system  and 
get  their  buy-in  on  the  final  model.  The  basic  flow  and  critical  points  are  drawn  in 
ARENA  14®  to  facilitate  ease  of  modification  while  going  over  the  process  with  SMEs 
at  Ogden  ALC. 

Figure  3  depicts  the  conceptual  flow  of  each  F-22  through  the  system  An  aircraft 
anives  for  input  into  the  system  and  all  maintenance  work  packages  are  loaded  into 
PDMSS  prior  to  starting  maintenance  on  the  aircraft.  In  the  arrival  process  for  the 
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conceptual  model  the  decision  is  made  on  the  type  of  aircraft  that  has  arrived  (based  on 
man-hour  requirements)  and  then  maintenance  actions  are  scheduled  by  gate. 


Figure  3:  Basic  Process  Flow 

Following  arrival  and  the  input  decision,  the  aircraft  is  towed  to  a  general 
maintenance  dock  for  scheduled  maintenance  for  Gates  1  through  4.  The  aircraft  enters 
Gate  1  on  the  schedule  and  the  focus  of  all  maintenance  efforts  are  on  completing  the 
maintenance  actions  assigned  to  Gate  1 .  Upon  completion  of  Gate  1  the  aircraft  enters 
into  Gate  2  on  the  schedule  and  all  maintenance  requirements  for  Gate  2  are  completed. 
The  same  steps  are  completed  through  Gate  4.  The  aircraft  is  moved  into  a  fuel  dock  at 
the  end  of  Gate  4  for  a  short  period.  SME  inputs  and  the  short  duration  of  fuel  dock 
usage  led  to  excluding  the  fuel  dock  from  consideration  because  it  is  not  considered  a 
critical  factor  for  inclusion  and  does  not  represent  a  present  or  future  constraint. 

Once  Gate  4  is  complete,  the  aircraft  is  moved  from  the  general  maintenance  dock 
to  the  LO  dock  for  Gates  5  and  6.  Gates  5  and  6  are  completed  in  the  same  way  as  the 
previous  four  gates  with  all  maintenance  actions  being  completed  prior  to  moving  into 
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the  subsequent  gate.  Upon  completion  of  Gate  6  the  aircraft  is  moved  from  an  LO  dock 
back  to  a  general  maintenance  dock  for  completion  of  Gate  7’s  maintenance 
requirements.  All  maintenance  requirements  for  the  aircraft  are  completed  before  exiting 
Gate  7.  Following  completion  of  all  maintenance  requirements  the  aircraft  exits  the 
system  and  returns  to  the  owning  organization.  After  conceptual  flow  mapping,  the 
detailed  simulation  model  requires  identifying  the  assumptions  that  are  made  prior  to 
beginning  development. 

Model  Assumptions,  Constraints,  and  Limitations 

A  simulation  model  is  an  abstraction  of  reality  and  therefore  assumptions  need  to 
be  made  because  the  complexity  of  the  F-22  Heavy  Maintenance  Modification  Program 
makes  it  unfeasible  to  include  every  aspect  of  reality  in  the  simulation  The  assumptions 
made  in  order  to  simplify  the  system  and  constraints  implemented  create  limitations  with 
the  model  and  associated  output  data.  Before  going  in  detail  on  the  steps  to  build  the 
model  the  assumptions,  limitations,  and  constraints  need  to  be  clearly  stated. 

Assumption  1:  SME  knowledge  is  considered  legitimate  and  the  insights  and  feedback 
on  system  operations  is  considered  to  be  accurate. 

Assumption  2:  All  adjusted  data  from  the  Clemson  Tool  is  representative  of  the  real 
inputs. 
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Assumption  3:  Every  employee  in  an  occupational  series  is  homogenous.  In  reality 
employees  may  have  varying  levels  of  experience  but  for  the  model  all 
employees  in  a  given  specialty  complete  maintenance  actions  at  the  same 
rate  and  quality.  Proficiency  impacts  are  reviewed  during  the  analysis 
portion  of  the  research. 

Assumption  4:  The  maintenance  man-hour  requirements  for  each  occupational  series 
required  in  a  given  flow  day  are  independent  of  subsequent  flow  days 
requirements. 

Assumption  5:  Leadership  and  production  supervisors  focus  maintenance  efforts  based  on 
first  in  first  out  prioritization  and  ensure  all  maintenance  actions  within  a 
gate  are  complete  prior  to  moving  to  the  next  gate. 

Assumption  6:  F-22  Heavy  Maintenance  Modification  Program  planners  and  operational 
wings  deconflict  arrivals  to  constrain  the  system  to  a  specific  number  of 
aircraft  or  WIP. 

Assumption  7:  The  number  and  type  ofpersonnel  seized  to  meet  man-hour  requirements 
for  a  given  flow  day  are  not  released  until  the  requirement  is  met.  Using 
the  aggregate  man-hour  requirements  vs.  tasks  eliminates  the  possibility  of 
personnel  being  reallocated  to  a  higher  priority  task  or  finishing  shorter 
tasks  and  moving  to  another  aircraft. 

Assumption  8:  The  four  aircraft  selected  for  simulation  have  man-hour  requirements  that 
are  representative  of  the  fleet. 

Assumption  9:  The  annual  throughput  is  based  on  225  workdays  per  year  and  model 
output  will  be  based  on  flow  days. 
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Limitation  1: 


Limitation  2: 


Limitation  3: 


Limitation  4: 


Constraint  1: 


Constraint  2: 


Constraint  3: 


Model  does  not  consider  external  factors  including  availability  of  parts  and 
specialties  outside  of  the  sixFWS  occupations  identified  on  the  critical 
path. 

The  aggregate  level  of  data  used  may  lessen  impacts  of  multi-skilling  due 
to  inflexibility  ofpersonnel  to  move  from  task  to  task. 

The  balanced  shift  modeling  approach  limits  direct  comparison  versus 
current  scheduling  policies.  For  this  research,  the  balanced  shift  approach 
is  defined  as  having  an  equal  number  of  personnel  on  each  shift  lor  each 
specialty. 

Assumption?  causes  model  to  hold  multi- skilled  resources  even  when  the 
primary  resource  becomes  available.  This  causes  an  overestimation  of 
multi-skilling  resources  needed  to  gain  a  certain  level  of  bene  fits.  In 
reality,  managers  could  reallocate  personnel  at  shift  change  or  during  the 
process  to  balance  the  current  workload. 

Initial  work  in  process  (WIP)  set  to  the  current  state  value  for  aircraft  WIP, 
doubling  aircraft  WIP  in  future  state. 

Maintenance  technicians  can  work  up  to  2  days  ahead  of  technicians  in 
differing  specialties  working  the  same  aircraft. 

The  maximum  number  ofpersonnel  in  a  specialty  seized  lor  work  on  one 
individual  aircraft  is  constrained  to  SME  defined  values. 
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Constraint  4: 


Constraint  5: 


Constraint  6: 


Constraint  7: 


The  maintenance  process  will  attempt  to  seize  enough  personnel  to 
complete  man-hour  requirements  for  a  specialty  in  1  shift,  not  to  exceed 
Constraint  3.  This  eliminates  seizing  16  personnel  to  complete  16  hour 
requirement  in  1  hour  (Ex.  10  hour  requirement  =  2  personnel  to  complete 
in  1  shift;  25  hour  requirement  =  5  personnel  to  complete  in  1  shift). 

Gate  4  will  take  no  less  than  12  days  to  complete.  This  constraint  reflects 
work  completed  by  specialties  outside  of  the  six  specialties  of  interest. 

Per  SME  feedback,  the  man-hour  requirements  do  not  reflect  the  flow 
days  it  takes  to  complete  an  aircraft  and  12  days  is  the  minimum  tune  to 
comp letc  the  gate. 

Gate  5  will  take  no  less  than  10  days  to  complete.  This  constraint 
represents  the  minimum  tune  that  an  aircraft  takes  to  complete  these  gates 
based  on  cure  time  associated  withLO  maintenance. 

Gate  6  will  take  no  less  than  8  days  to  complete.  This  constraint  represents 
the  minimum  time  that  an  aircraft  takes  to  complete  these  gates  based  on 
cure  time  associated  with  LO  maintenance. 


Model  Description 

ARENA  14®  is  the  software  chosen  for  use  in  this  research.  It  is  a  commercial 
simulation  software  in  which  the  Air  Force  has  several  licenses  and  currently  uses  in  a 
variety  of  decisional  and  educational  capacities.  The  software  requires  minimal  coding 
and  includes  built  incapabilities  that  facilitate  timely  development  of  comp  lex  simulation 
models.  The  simulation  model  developed  in  ARENA  14®  is  based  on  the  conceptual 
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model  previously  described  in  Figure  3.  The  following  sections  briefly  describe  the 
model  developed.  See  Appendix  A  for  the  full  model  description  and  development. 

The  F-22  (entity  of  interest)  flows  through  eight  main  sub  models  including 
Characteristics  and  a  separate  sub  model  for  each  of  the  Gates  1  through  7  as  depicted  in 
Figure  4:  Top  Level  Model.  For  the  current  state  baseline  model  the  aircraft  arrives 
every  19  days  based  on  the  real  system  target  annual  throughput  with  225  workdays  per 
year.  Once  the  aircraft  is  processed  in  all  eight  main  sub  models  it  exits  the  system  and  is 
disposed  of  in  the  Depart  process.  The  variable  views  showing  zeros  track  the  variables 
indicated  by  the  title  above  each  one  as  the  model  runs. 


Top  Level  Model 


«  o  o  o  e  o 


PlaneNumber 


o 


Process  Time 


0.0 


Current  WIP 

0~~ 


Total  A/C 

(T 


Flow  Days  Average  Flow  Days 


Figure  4:  Top  Level  Model 
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Characteristics  Sub  Model 


After  entity  creation  the  aircraft  flows  into  the  Characteristics  sub  model  as  seen 
in  F  igure  5  for  decisions  on  the  type  of  aircraft  arriving  and  to  “deconflict”  arrivals  based 
on  the  constraint  to  have  only  a  certain  number  of  aircraft  within  the  system  at  any  tune. 


Figure  5:  Characteristics  Sub  Model 


Upon  entering  the  Characteristics  sub  model  seen  in  Figure  5,  the  aircraft  flows 
through  the  Awaitinglnput  module  and  is  held  until  the  number  of  work- in-process  (WIP) 
aircraft  in  the  system  is  less  than  the  maximum  WIP  constraint  using  Equation  5.  The 
WIP  Constraint  variable  is  used  as  the  maximum  WIP  constraint  and  the  value  is  initially 
set  to  reflect  the  current  constraint  of  the  real  system 

(Gatel.WIP  +  Gate2.WIP  +  Gate3.WIP  +  GateA.WIP  + 

(5) 

GateS.WIP  +  Gate6.WIP  +  Gate? .WIP )  <  WIP_Constraint 

The  aircraft  then  flows  into  the  Assign  ID  process  and  is  assigned  a  unique 
identification  (ID)  number,  a  unique  ID  variable,  and  a  plane  picture  for  animation 
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purposes.  Next,  the  aircraft  is  counted  as  a  depot  input  and  the  decision  is  made  on  the 
type  ofplane  arriving  based  onrandom  chance  with  a  certain  probability  of  the  four 
different  types  of  arrivals  being  selected.  The  decide  module  uses  ARENA’S  standard 
random  number  generation  stream  and  is  the  only  stochastic  input  in  the  model.  The 
random  selection  of  the  four  types  of  aircraft  induces  man-hour  requirement  variability 
into  the  system  and  is  a  critical  factor  to  accurately  reflect  the  variability  of  real  system 
inputs.  The  four  plane  types  are  used  in  Gates  1  through  7  sub  models  to  assign  man¬ 
hour  requirements  for  maintenance  processing  times.  The  decision  on  the  number  of 
plane  types  and  selection  of  hourly  requirements  is  discussed  later  in  the  Inputs  portion  of 
this  chapter.  Following  the  decision  on  plane  type,  the  aircraft  exits  the  Characteristics 
sub  model  for  Gate  1  and  is  defined  as  work- in-process  (WIP). 

Gate  1  Sub  Model 

Transformations  from  inputs  into  outputs  through  the  completion  of  maintenance 
hour  requirements  begin  in  Gate  1.  The  previous  sub  model  is  used  to  determine  the 
plane  type  or  P la neNum  attribute  based  on  the  four  possible  plane  types  and  then 
constrains  the  system  to  a  defined  WIP  value.  Gate  1  uses  the  attributes  defined  in  the 
Characteristics  sub  model  for  decisions  on  processing  times  per  specialty  as  the  aircraft 
moves  from  one  day  to  the  next.  The  Gate  1  sub  model  is  broken  down  into  six  main 
processes,  two  main  sub  models  with  several  record  functions  used  to  tally  statistics 
throughout  the  gate.  Appendix  A  shows  the  full  layout  of  Gate  1  with  in-depth 
discussions  on  the  modules  and  functions.  The  following  sections  discuss  the  Gate  1  Sub 
Model  in  two  sections,  shown  in  Figure  6. 
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The  first  half  of  the  Gate  1  discussion  centers  on  the  modules  between  points  A 
and  B  in  Figure  6.  The  second  half  will  center  on  the  processes  betweenpoints  B  and  C. 
The  researcher  begins  the  discussion  with  the  first  processes  and  the  ProcessGl  sub 
model  in  Gate  l,  depicted  in  Figure  7:  Gate  1  Sub  Model  (a). 
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The  first  process  upon  entering  Gate  1,  GatelStart,  assigns  the  gate  start  time  as 
the  aircraft  enters  the  gate.  Next,  the  aircraft  is  separated  into  six  different  entities  using 
five  separate  blocks  and  then  flows  into  separate  paths  representing  the  six  FWS 
specialties  of  interest. 

In  queuing  theory  terms,  each  path  represents  a  different  type  of  server  required  to 
complete  a  specific  type  of  maintenance.  The  system  has  six  types  of  servers  (FWS 
maintenance  specialties)  that  can  do  maintenance  and  using  a  different  path  design  allows 
resource  pooling  based  on  the  type  of  server.  When  multi-skilling  is  introduced  the 
server  is  able  to  complete  maintenance  hours  associated  with  multiple  resource  pools 
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versus  just  one.  The  paths  are  replicas  of  each  other  with  minor  equation  differences. 
Only  the  AV  (avionics)  path  is  discussed  in  detail  to  avoid  duplication  and  the  differences 
of  the  paths  are  subsequently  detailed  in  the  full  model  description  in  Appendix  A.  For 
consistency  purposes,  the  separated  aircraft  is  referred  to  as  entities  and  again  described 
as  an  aircraft  when  they  are  joined  back  together. 

The  separated  entity  flows  into  an  assign  module,  Gatel  AV  in  which  a  global 
variable  (AV  Available)  and  attribute  (Counter)  are  assigned.  The  Counter  attribute  is 
given  a  value  +1  and  adds  one  to  the  count  every  time  the  entity  flows  through  the  assign 
module.  The  AV  Available  variable  assigns  the  current  value  based  on  the  number  of 
avionics  technicians  available  in  the  set  of  resources  called  AVIONICS  using  Equation  6. 

A  V  _  Available  =  Number  Available  -  NumberBusy  (6) 

The  AVIONICS  set  includes  the  avionics  personnel  (AV  resource)  and  a  separate 
multi-skill  resource  for  each  of  the  five  other  specialties  paired  with  AV  (APAV, 
AS_AV,  etc).  The  benefits  of  using  a  set  are  that  sets  allow  multiple  resource  types  to  be 
seized  to  complete  processing  (maintenance  requirements).  This  is  important  with  the 
introduction  of  a  multi-skilled  workforce  because  it  allows  multiple  resource  pools  to 
complete  maintenance  man-hour  requirements. 

VBA  Block 

Following  the  assign  module  the  entity  advances  to  a  visual  basic  application 
(VBA)  block.  The  block  runs  custom  VBA  code  that  first  finds  the  path  for  an  excel 
document  (Model  Input  Data.xls)  containing  man-hour  requirements  by  specialty  and 


49 


flow  day  for  the  aircraft  inputs  selected.  The  document  needs  to  be  open  prior  to  running 
the  model  to  minimize  runtime  associated  with  continually  opening  the  document. 

Next,  the  VBA  code  looks  to  the  tab  for  one  of  the  aircraft  inputs  based  on  the 
PlanNum  attribute.  Then  it  starts  at  the  first  day  of  requirements  for  the  given  specialty 
path  the  aircraft  is  in  and  assigns  the  man-hour  requirement  value  to  the  ProcessTime 
attribute.  Concurrently  the  code  references  a  different  row  and  cell  for  the  following 
flow  day  to  identify  the  start  of  the  next  gate.  If  the  next  day  is  the  start  of  the  next  gate 
the  VBA  code  assigns  a  ContinueFlag  attribute  value  of  zero  to  reflect  the  last  day  of  the 
gate.  Once  the  ProcessTime  and  ContinueFlag  values  are  assigned,  the  aircraft  proceeds 
to  a  sub  model  called  AVProcessGl. 

AVProcessGl  Sub  Model 

The  AV  ProcessGl  sub  model  performs  two  main  functions  1)  it  decides  on  the 
number  of  maintenance  personnel  to  seize  in  order  to  fulfill  daily  man-hour  requirements 
for  the  aircraft  and  2)  it  seizes  the  aircraft  and  personnel  resources  for  the  required 
numbers  of  hour  to  meet  the  days  maintenance  requirements.  The  path  through  the  four 
processes  in  the  sub  model  is  depicted  Figure  8:  AV  ProcessGl  (Gate  1  Sub  Model 
between  points  1  and  2. 
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Figure  8:  AV  ProcessGI  (Gate  1  Sub  Model) 


The  first  step  is  deciding  on  whether  the  aircraft  has  maintenance  requirements  for 
the  current  flow  day.  The  TimeCheckAVIGl  process  decides  if  maintenance 
requirements  exist  on  the  aircraft  for  the  day.  If  the  ProcessTime  attribute  is  greater  than 
zero  then  the  aircraft  flows  on  to  AVI  Number  process,  otherwise  it  bypasses  the  other 
processes  and  exits  the  sub  model.  The  bypass  is  important  because  the  aircraft  could  be 
held  in  the  queue  to  be  processed  in  subsequent  blocks  even  though  it  had  no 
requirements  for  the  day.  Once  the  decision  is  made  to  continue  or  bypass,  the  entities 
requiring  maintenance  flow  into  the  AVI  Number  sub  model  shown  in  Figure  9  for 
resource  allocation  decisions. 
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The  basic  function  of  AVI  Number  sub  model  is  to  assign  the  number  of 
personnel  to  be  seized  to  meet  man-hour  requirements  for  the  day  based  on  the  number  of 
personnel  available.  If  none  are  available  it  holds  the  aircraft  in  the  sub  model  until 
personnel  become  available.  Before  any  decisions  are  made  the  aircraft  is  delayed  for 
one  minute  in  the  AV  Delay  SeizeDecisionl  sub  model  so  previously  held  aircraft  can 
exit  the  sub  model  when  personnel  become  available.  The  delay  is  important  because  it 
allows  aircraft  held  in  the  sub  model  to  clear  once  personnel  become  available  prior  to 
the  next  aircraft  initiating  resource  allocation  decisions,  preserving  the  first- in- first- out 
priority  processing  of  the  model.  The  aircraft  then  flows  through  several  decision  points 
in  order  to  avoid  allocating  more  personnel  than  SMEs  defined  as  the  maximum  number 
of  personnel  that  would  be  assigned. 
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Additionally,  it  constrains  the  number  of  personnel  assigned  to  complete 
maintenance  actions  to  no  more  than  are  required  to  complete  the  maintenance  hours 
required  in  one  shift.  Further  defined,  the  constraint  assigns  a  value  for  the  maximum 
number  of  personnel  required  to  complete  the  flow  day  man-hour  requirements  without 
tailing  below  the  minimum  time  the  aircraft  should  be  in  work  each  flow  day  as 
identified  by  SMEs.  Following  the  decision  on  the  number  ofpersonnel  needed  to 
complete  the  maintenance  requirements,  the  entity  flows  to  the  AVDIProcess. 

AVDl  Process 

The  AVDl  process  is  the  heart  of  this  simulation  research.  The  aircraft  flows 
into  the  process  and  is  delayed  while  maintenance  man-hour  requirements  are  met  lor  the 
day.  This  process  and  its  sister  processes  in  the  other  specialty  paths  are  the  only  process 
modules  that  convert  inputs  into  outputs  within  the  system  The  input  is  considered  an 
aircraft  with  maintenance  requirements  and  an  output  is  considered  an  aircraft  with  all 
maintenance  man-hour  requirements  met.  All  other  processes  up  to  this  point  are  used  to 
assign  attributes  and  variables  to  facilitate  this  process. 

The  process  uses  a  standard  type  of  module  with  a  Seize  Delay  Release  action  and 
assigns  resources  by  Set.  The  set  name  is  AVIONICS_AV  and  includes  the  AV  resource 
and  the  five  additional  multi-skill  resources.  The  quantity  used  is  the  AV  SeizeNum 
attribute  assigned  in  the  AVI  Number  sub  model,  discussed  fully  in  Appendix  A.  The 
selection  rule  is  Preferred  Order  and  the  order  is  set  for  all  runs  subsequent  to  the  baseline 
ran  with  the  priority  of  first  seizing  resources  that  have  lower  utilization  rates  in  the 
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baseline  outputs.  Identifying  the  quantity  of  resources  (personnel)  seized  is  a  critical  step 
before  moving  onto  the  calculation  of  the  processing  (delay)  tune  value. 

The  amount  of  time  the  aircraft  is  delayed  (processing  tune)  is  based  the  man¬ 
hours  required  for  AV  on  the  current  flow  day  (Process  Time),  the  number  of  personnel 
assigned  to  complete  maintenance  (AV  SeizeNum),  and  an  efficiency  factor  for  the  pool 
of  resources  used  (EfficiencyFactorAV).  Equation  7  depicts  the  function  used  to  decide 
the  value  of  the  process  delay. 

ProcessTime/  (  EfficiencyFactorAV  *  AV  SeizeNum )  (7) 

The  efficiency  factor  is  used  to  adjust  the  efficiency  of  the  resource  pool  with 
multi-skilling.  The  initial  value  of  the  resource  pool  is  one  and  can  be  adjusted  down 
(.99,  .98,  etc.)  to  depict  a  loss  in  skill  or  efficiency.  Levien(2010)  multiplies  processing 
times  by  a  cross  training  factor  in  his  multi-skill  research  on  the  KC-135  PDM  process  in 
order  to  emulate  longer  task  durations  associated  with  multi-skilled  employees 
completing  maintenance  tasks.  The  efficiency  factor  in  this  research  mathematically 
produces  similar  results  by  increasing  task  times  as  efficiency  (task  proficiency) 
decreases.  However,  multi-skilling  an  employee  stipulates  that  the  employee  is  fully 
qualified  at  the  journeyman  level  on  both  skill  sets  (Federal  Service  Impasses  Panel, 

1997)  and  therefore  equally  proficient.  The  baseline  model  and  experimental  runs  are 
initially  run  with  an  efficiency  factor  of  one.  The  analysis  section  of  this  chapter  explains 
the  sensitivity  analysis  methods  used  to  gauge  the  impacts  on  model  outputs  of 
decreasing  skill  within  the  labor  pool.  Once  the  maintenance  man-hours  are  met  for  the 
current  flow  day  the  aircraft  is  released  and  proceeds  through  modules  that  record  several 
variable  values,  then  the  entity  exits  the  AV_DlProcess. 
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The  next  few  paragraphs  discuss  the  second  half  of  the  Gate  1  sub  model, 
covering  the  last  sub  modeland  process  modules  in  Gate  1  (depicted  inFigure  10). 


In  synopsis,  the  second  half  of  the  gate  restrains  or  aligns  the  six  identical  entities 
in  order  to  stay  within  a  certain  number  of  flow  days  of  each  other.  It  then  re- loops  the 
entities  back  to  the  beginning  of  the  gate  when  the  entity  requires  more  flow  days  within 
the  gate  and  combines  the  entities  back  into  one  aircraft  before  allowing  it  to  exit  the 
current  gate.  The  mechanism  aligning  flow  days  by  identical  entities  is  the  first  process 
in  the  second  half  of  the  gate  and  occurs  within  the  sub  model  GatelMatch. 

GatelMatch 

The  first  process  in  the  second  half  of  the  gate  is  the  GatelMatch  sub  model 
described  fully  in  Appendix  A.  This  process  is  important  because  it  restricts  the  six 
duplicates  of  one  aircraft  to  within  a  certain  number  of  flow  days  of  each  other  based  on 
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SME  consultations.  The  LoopConstraint  is  the  variable  created  and  used  to  reflect  the 
SME  defined  constraint  for  how  far  ahead  maintenance  specialties  can  work  from  other 
specialties  working  the  same  aircraft.  Functionally,  this  process  realigns  all  six 
specialties’  entities  to  the  same  flow  day  and  does  so  every  so  many  flow  days  as  defined 
by  the  LoopConstraint.  Once  the  flow  days  are  realigned  the  entity  flows  out  of  the  sub 
model  to  another  decide  module  in  Gate  1. 


ContinueAV?  Module 

The  next  decide  module  Continue  A  V?  decides  whether  or  not  the  entity  has  more 
flow  day  requirements  in  the  current  gate.  The  parameters  displayed  in  Figure  1 1  re- loop 
the  entity  back  to  GatelAV  module  if  the  ContinueFlag  attribute  is  one.  The  aircraft 
then  recompletes  the  Gate  1  AV  path.  Recall  that  the  VBA  block  assigns  a  value  of  zero 
to  the  ContinueFlag  if  the  next  flow  day  is  the  start  of  the  next  gate.  When  the  value  is 
zero  and  the  entity  enters  the  Continue  AV?  module,  the  condition  is  false  and  the  entity 
is  directed  to  the  exit  path  of  the  Gate  1  sub  model. 


Figure  11:  Continue  AV? 
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Gate  1  Exit  Path 


The  exit  path  of  each  gate  routes  through  five  modules  before  exiting  Gate  1 .  The 
modules  include  a  batching  module,  three  record  modules,  and  an  assign  module  as 
sho  wn  in  F  igure  1 2 . 


The  important  characteristic  of  the  exit  path  is  that  entity  flows  to  the 
Gatel  Batch  module  and  is  held  until  the  other  five  entities  with  the  same  1DN umber 
arrive.  Then  entities  are  combined  back  into  one  aircraft  and  it  is  released  to  the  final  two 
record  modules  and  an  assign  module.  Gates  1  through  7  are  almost  identical  with  only 
minor  differences  in  regards  to  hold  functions,  record  modules,  and  dock  seize  modules. 
For  in-depth  description  of  each  difference,  refer  to  the  full  model  description  in 
Appendix  A. 

The  model  of  an  aircraft  flowing  through  the  F-22  Heavy  Maintenance 
Modification  Program  is  discussed  in  the  previous  sections  with  a  focus  on  the  process  of 
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transforming  an  input  aircraft  into  an  output  aircraft  with  all  maintenance  requirements 
completed.  In  order  for  the  model  to  transform  an  input  aircraft  into  outputs,  the  inputs 
for  the  model  must  be  identified. 

Inputs 

Resources 

The  resource  inputs  into  the  model  include  personnel  from  the  six  FWS 
specialties  of  interest,  general  maintenance  docks,  and  low  observable  (LO)  maintenance 
docks.  The  general  maintenance  and  LO  docks  are  set  at  the  current,  then  future  state 
levels  based  on  the  current  and  projected  number  available.  The  personnel  inputs  require 
further  discussion  on  their  inputs. 

Currently,  the  F-22  Heavy  Maintenance  Modification  Program  operates  on  a 
three-shift  schedule  with  only  LO  personnel  on  mid  shift.  During  this  research,  a 
limitation  was  identified  with  the  scheduling  function  in  ARENA  14®,  hindering  the 
ability  to  simulate  differing  resource  levels  across  the  three  shifts.  The  scheduling 
function’s  preempt  rule  only  preempts  one  entity’s  resources  when  multiple  entities  are 
being  processed  within  a  model.  The  schedule  can  change  to  reflect  no  resources 
available  but  ARENA  14®  allows  the  model  to  continue  to  process  the  entity,  even  with 
no  resources  available.  This  causes  the  model  to  allow  for  more  labor  hours  than  should 
be  available  and  distorts  the  scheduled  resource  utilization  statistics. 

However,  the  current  proposal  is  for  balanced  shifts  with  the  addition  of  the  future 
workload.  Balanced  shifts  reflect  equal  values  of  personnel  for  each  of  the  three  shifts 
including  days,  swings,  and  midnight  shifts.  After  consulting  with  F-22  Heavy 
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Maintenance  Modification  Program  leadership,  the  decision  is  made  to  simulate  using  the 
balanced  shift  approach  for  both  current  and  future  state,  allowing  for  direct  comparisons 
across  the  models  and  eliminating  the  limiting  factor  associated  with  ARENA  14®’ s 
scheduling  (preempt)  function. 

The  personnel  capacity  for  each  specialty  is  set  to  the  value  of  one  shift  of 
personnel.  The  model  hours  per  day  value  is  adjusted  to  reflect  the  duration  of  three 
shifts  worth  of  labor  hours,  mimicking  the  number  of  hours  that  personnel  would  be 
available  to  complete  maintenance  in  a  given  day.  This  method  reflects  the  same 
personnel  and  labor  hours  available  by  scheduling  three  equal  personnel,  equal  duration 
shifts.  The  only  difference  is  the  shift  change  is  removed  and  the  personnel  continue  on 
the  job  instead  of  instantaneously  stopping  maintenance,  changing  personnel,  and 
restarting  the  same  maintenance  (processing)  with  the  same  type  of  person  or  resource. 

The  personnel  values  are  based  on  SME  feedback  and  reflect  maintenance 
technicians  scheduled  in  June  of2013.  For  simplification  of  cost  calculations,  the 
decision  is  made  to  consider  all  scheduled  technicians  as  WG-10  employees  even  though 
the  schedule  includes  some  WL  employees.  The  personnel  numbers  do  not  include 
personnel  recently  hired  and  awaiting  clearances. 

For  the  current  state,  one  shift  worth  of  personnel  for  each  specialty  are  input  into 
the  model  based  on  the  values  in  Table  1.  The  future  state  doubles  the  number  of 
personnel  available  in  each  of  the  specialties.  Once  the  decision  on  resource  inputs  is 
made,  the  next  step  is  the  decision  on  data  inputs. 
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Table  1:  Current  State  Personnel  Resources 


Current  State  Balanced  Schedule 

Manning  PerSkill 

Days 

Swings 

Mids 

Total 

AP(LO) 

32 

32 

32 

96 

AS  (Sheet  Metal) 

18 

18 

18 

54 

AR  (Electrician) 

4 

4 

4 

12 

AG gen  (A/C  General) 

7 

7 

7 

21 

AT  (Fuels) 

3 

3 

3 

9 

AV  (ATE/Radar) 

2 

2 

2 

6 

Total 

198 

Data 

The  model  scope  decision  led  to  using  the  Clemson  scheduling  tool  for  input  data 
for  the  simulation  but  does  not  lead  to  a  conclusion  on  the  aircraft  data  sets  to  use  from 
within  the  tool.  In  the  next  couple  of  sections  the  decisions  on  the  type  of  data  to  input 
from  within  the  Clemson  tool,  the  aircraft  data  set  sample  size,  and  the  percentage  of 
arrivals  for  each  type  of  aircraft  are  discussed. 

The  first  decision  is  on  the  type  of  data  to  input  or  pull  into  the  C  lemson 
scheduling  tool.  The  tool  can  import  forecasted  man-hour  requirements  per  specialty  per 
day  based  on  the  standard  man-hours  scheduled  to  complete  the  maintenance  packages 
assigned  or  it  can  pull  the  actual  man-hours  per  specialty  per  day  that  were  used  on  an 
aircraft  that  has  been  previously  completed.  The  decision  is  made  to  use  standard  man¬ 
hours  required  to  complete  an  aircraft  after  comparing  forecasted  (standard)  and  actual 
man-hour  requirements  across  several  aircraft.  The  review  is  completed  and  discussed 
with  SMEs  and  the  Clemson  scheduling  tool  does  not  pull  in  the  O  &  A  hours  and 
therefore  under  reflects  the  man-hour  requirements.  However,  the  standard  data 
overestimates  AP  man-hour  requirements  and  the  adjustments  needed  to  compensate  for 
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the  overestimation  is  discussed  in  detail  during  a  comparative  analysis  in  later 
paragraphs.  The  decision  to  use  standard  man-hour  requirements  allows  the  researcher  to 
proceed  to  the  next  decision,  selection  of  the  sample  size  of  aircraft  data  for  input  use. 

The  selection  of  sample  size  is  straightforward  due  to  recent  changes  to  the  F-22 
Heavy  Maintenance  Modification  Program.  The  number  of  aircraft  data  sets  to  choose 
from  is  limited  due  to  a  change  of  their  processes  to  the  gate  emphasis  in  July  of2012. 
Aircraft  that  were  completed  prior  to  that  date  are  excluded  because  some  tasks  were 
realigned  to  reflect  the  milestone  focus  or  to  facilitate  improvements  identified  by  the 
new  focus.  This  restricts  the  potential  data  sets  to  seven  aircraft  that  have  been 
completed  under  the  new  system  at  the  time  of  model  development. 

Further  narrowing  the  data  sets  available  is  the  impact  of  minimizing  and  in  some 
cases  allowing  no  overtime  at  the  end  ofFY  2013  due  to  sequestration.  The  aircraft 
comp letcd  during  this  period  are  eliminated  to  ensure  the  data  sets  are  consistent  in  terms 
of  work  conditions,  shifts,  and  overtime  usage.  These  changes  narrowed  the  potential 
sample  size  from  seven  aircraft  to  six  aircraft. 

Four  aircraft  data  sets  are  selected  for  inputs  from  the  six  data  sets  available  after 
discussions  with  SMEs  and  due  to  availability  of  flow  day  calculations  and  actual  man¬ 
hour  requirements  compiled  by  the  F-22  Flight  planners.  The  planners’  flow  day 
calculations  include  all  of  the  four  aircraft  selected  and  allow  for  potential  validation 
comparisons  on  a  one  to  one  basis.  Furthermore,  the  other  two  aircraft  data  sets  reflect 
man-hour  requirements  that  are  similar  to  two  of  the  aircraft  selected.  Additionally,  the 
four  aircraft  data  sets  closely  reflect  the  projected  aircraft  man-hour  requirements  for  FY 
2014  with  man-hour  requirements  between  16,000  and  24,000  hours. 
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During  a  comparative  analysis  of  the  total  hours  used  on  the  aircraft  selected 
versus  the  standard  projected  man-hour  requirements,  the  researcher  found  that  the 
standard  hours  significantly  over- forecasted  the  man-hour  requirements  for  AP  (Low 
Observable)  technicians  in  three  of  the  four  aircraft.  Plane  2  AP  hours  are  within  four 
percent  of  actual  requirements  but  the  other  aircraft  were  over  forecast  by  up  to  40 
percent.  In  order  to  assist  in  identifying  the  most  likely  area  of  deviation,  the  researcher 
plotted  the  AP  cumulative  forecast  hours  for  the  four  aircraft  against  each  other  as  seen  in 
Figure  13.  The  figure  identifies  a  few  key  points: 


Plane  1 
Plane  2 
Plane  3 
Plane4 


Figure  13:  AP  Cumulative  Standard  Hours 

First,  there  are  clearly  delineated  increases  in  AP  hours  as  expected  for  Gates  5 
and  6.  Second,  it  confirms  that  most  AP  man-hour  requirements  exist  in  Gates  5  and  6 
and  reflects  the  real  system  as  observed  by  the  researcher  and  SMEs.  Third,  the  large 
deviation  in  hourly  requirements  between  aircraft  begins  in  Gate  6.  F inally,  the 


AP  Cumulative  Standard  Hours  Start 


Flow  Days 
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maximum  actual  AP  (LO)  hours  used  on  any  of  the  six  aircraft  in  the  computations  by 
F-22  Planners  are  below  12,000  hours,  however  F igure  13  depicts  two  aircraft  well  over 
12,000  hours.  Further  analysis  of  the  daily  forecast  hours  confirms  that  extreme  man¬ 
hour  requirements  exist  in  the  17  days  of  Gate  6.  By  extreme,  the  researcher  means 
hourly  requirements  for  a  day  that  need  14  days  to  complete  given  current  constraints. 
Discussions  with  SME’s  on  this  issue  further  supports  the  finding  that  Gate  6  hourly 
requirements  are  over-estimated.  The  Gate  6  standard  hourly  requirements  for  AP  are 
currently  being  adjusted  by  the  F-22  Heavy  Maintenance  and  Modification  Program  to 
accurately  reflect  current  requirements. 

In  order  to  solve  this  problem  the  decision  is  made  to  adjust  Gate  6  AP  over¬ 
estimated  man-hour  requirements  to  more  accurately  reflect  the  true  system  man-hour 
requirements.  Hourly  adjustments  are  made  on  three  of  the  four  aircraft  and  are 
conservative  in  order  to  adjust  the  over-estimates  to  be  more  representative  of  the  real 
data  while  not  trying  to  align  the  data  exactly  to  the  real  aggregate  man-hour 
requirements.  The  adjustment  is  made  by  subtracting  two  thousand  five  hundred  hours 
fromPlane  1  and  3  an  average  of  147  hours  from  each  of  the  17  days  in  Gate  6,  because 
they  were  more  than  three  thousand  hours  over-estimated.  The  researcher  subtracts  one 
thousand  seven  hundred  hours  from  Plane  4,  an  average  of  100  hours  from  each  of  the  17 
days  in  Gate  6,  because  it  was  almost  two  thousand  hours  o veres thnated.  Plane  2  is 
unaltered  because  the  standard  hours  were  more  closely  aligned  with  the  real  cumulative 
AP  requirements.  Table  2  identifies  the  differences  between  the  standard  total  hours 
forecasted  versus  the  actual  aggregate  hours  used  and  then  compares  the  adjusted 
standard  total  hours  versus  the  actual  aggregate  hours  used.  Note  that  Plane  2  hours  are 
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provided  for  comparison  against  the  other  three  aircraft  but  the  man-hours  from  standard 


to  adjusted  for  Plane  2  remains  unchanged. 

Table  2:  AP  Hours  Standard  vs  Actual 


AP  Standard  /  Adjusted  Standard  Hours  vs  Actual  Hours 


Aircraft 

Category 

Input  Hrs 

Actual  Hrs 

Difference 

Deviation 

Plane  1 

Standard  AP 

14090.2 

10987.08 

3103.12 

28% 

Adjusted  AP 

11546.6 

10987.08 

559.52 

5% 

Plane  2 

Standard  AP 

11976 

11487.79 

488.21 

4% 

Adjusted  AP 

11976 

11487.79 

488.21 

4% 

Plane  3 

Standard  AP 

13405 

9581.39 

3823.61 

40% 

Adjusted  AP 

10906.7 

9581.39 

1325.31 

14% 

Plane  4 

Standard  AP 

10122.9 

8197.75 

1925.15 

23% 

Adjusted  AP 

8422.9 

8197.75 

225.15 

3% 

The  adjustments  do  not  eliminate  the  over-estimation  but  the  adjusted  data  more 
accurately  reflects  the  true  system  requirements  for  AP  man-hours.  Furthermore,  Figure 
14  shows  that  the  variability  in  total  AP  man-hour  requirements  is  retained  between 
aircraft  while  eliminating  the  previous  over-estimate  values  near  14  thousand  man-hours. 
The  adjustment  is  not  a  perfect  solution  but  it  is  the  best  course  of  action  given  the  data 
available  for  comparison.  SMEs  verified  the  adjustments  were  accurate  compared  to  the 
real  system,  noting  that  they  are  currently  adjusting  the  AP  forecast  standard  hours  to 
more  accurately  reflect  the  true  requirements. 
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Plane  1 
Plane  2 
Plane  3 
Plane  4 


Figure  14:  AP  Cumulative  Adjusted  Standard  Hours 

The  five  other  critical  path  maintenance  specialties’  man-hour  requirements 
reflect  percentage  differences  in  man-hour  requirements  similar  to  the  adjusted  AP 
values.  No  adjustments  are  made  to  these  specialties  because  a  targeted  adjustment  of 
their  hourly  requirements  is  not  feasible  given  the  data  currently  available  and  the  lact 
that  their  standard  man-hour  forecast  is  much  closer  to  the  true  system  man-hour 
requirements  than  the  original  AP  standard  man-hours.  Once  the  input  data  is  selected 
and  adjusted,  the  next  step  is  the  decision  on  the  aircraft  type  to  enter  the  system. 

The  next  step  is  to  decide  the  input  percentage  for  each  of  the  four  types  of 
aircraft  that  arrives  into  the  current  system.  The  man-hour  requirements  for  aircraft  that 
have  a  projected  induction  in  2014  are  analyzed  to  aid  in  the  decision  on  the  input 
percentages  for  each  of  the  four  aircraft  data  sets.  The  aircraft  are  separated  into  four 
bins  based  on  their  projected  man-hour  requirements  compared  to  the  four  unadjusted 
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aircraft  data  sets  that  were  selected  for  input.  The  unadjusted  standard  man-hour 
requirement  is  used  because  the  assumption  can  be  made  that  future  projections  will  need 
similar  adjustments,  therefore,  unadjusted  data  sets  accurately  reflect  unadjusted 
projections  for  2014.  The  bins  reflect  the  percentage  of  aircraft  that  fell  within  each  of 


the  four  data  sets  as  shown  in  Figure  15. 
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Figure  15:  2014  Projected  F-22  Hours  (Bins) 


The  percentages  seen  in  Table  3  are  then  input  into  the  model  for  the  values  of  the 
ProbPlanel,  ProbPlane2,  and  ProbPlane3  variables.  Plane  4  does  not  need  to  be  specified 
because  all  others  (those  not  in  the  first  three)  travel  the  Plane  4  path,  reflecting  the  last 
33.33  percent  of  input  aircraft. 
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Table  3:  inputs-2014  Projection  Bins 


Plane 

Bin 

Bin  % 

Cumulative 

% 

Plane  4 

17363 

4 

33.33% 

33.33% 

Plane  2 

20127 

2 

16.67% 

50.00% 

Plane  3 

22399 

3 

25% 

75.00% 

Plane  1 

23968 

3 

25% 

100.00% 

The  future  state  model  input  percentages  use  the  same  methods  as  the  current 
state  model.  The  additional  aircraft  per  year  representing  the  addition  of  the  Palmdale 
depot  maintenance  activities  reflect  higher  man-hour  requirements  than  the  existing  F-22 
depot  maintenance  workload  at  Ogden.  Additionally,  the  previously  identified  current 
state  aircraft  inputs  remain  representative  of  half  the  expected  workload  in  the  future 
state.  After  reviewing  the  F-22  Depot  Flow  Plan,  the  additional  aircraft  are  binned  with 
the  previous  aircraft,  allocating  40  percent  of  the  aircraft  into  the  Plane  3  bin  and  60 
percent  into  the  Plane  1  bin  as  shown  in  Figure  16. 


67 
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Figure  16:  Future  State  Projected  F-22  Hours 


The  percentages  seen  in  Table  4  are  then  input  into  the  future  state  model  for  the 
values  of  the  ProbPlanel,  ProbPlane2,  and  ProbPlane3  variables.  Plane  4  does  not  need 
to  be  specified  because  all  others  (those  not  in  the  first  three)  travel  the  Plane  4  path, 
reflecting  the  last  18.18  percent  of  input  aircraft. 


Table  4:  Future  State  Input  Percentages 


Plane 

Bin 

Frequency 

Bin  % 

Cumulative 

% 

Plane  4 

17363 

4 

18.18% 

18.18% 

Plane  2 

20127 

2 

9.09% 

27.27% 

Plane  3 

22399 

7 

31.82% 

59.09% 

Plane  1 

23968 

9 

40.91% 

100.00% 

Recall  that  the  custom  VBA  code  previously  described  pulls  the  input  data  into 
the  model  for  simulation  runs  based  on  the  plane  type  assigned  to  each  aircraft.  The 
adjusted  standard  man-hour  requirement  data  sets  for  the  four  input  aircraft  are  copied 
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into  a  spreadsheet  labeled  Model  Input  Data.xls.  Due  to  the  access  requirements 
associated  with  F-22  data,  the  sample  Model  Input  Data  spreadsheet  in  Appendix  A 
reflects  mock  data.  After  inputs,  the  next  methodology  topic  for  description  is 
verification  and  validation  of  the  model. 

NOTE:  The  input  spreadsheet  must  be  saved  as  an  .xls  file  or  ARENA  14®  will  not  interface 
with  it  correctly. 

Model  Verification  &  Validation 

Verification  and  validation  of  a  simulation  model  ensures  the  design  and  function 
accurately  represents  the  behaviors  of  the  real  system.  Recall  that  according  to  Carson 
(2005),  “The  result  of  the  V  &  V  phase  is  a  verified,  validated  model  that  is  judged  to  be 
accurate  enough  for  experimentationpurposes  over  the  range  of  system  designs 
contemplated”  (p.  21).  For  this  reason,  the  simulation  model  is  built  using  a  piecewise 
tashion  with  each  verification  and  validation  method  being  used  continually  throughout 
the  process. 

Verification 

Model  verification  serves  to  ensure  the  operational  model  accurately  reflects  the 
conceptual  model  and  real  system  of  interest.  Building  the  simulation  in  a  piecewise 
tashion  and  conducting  verification  prior  to  duplicating  the  function  was  key  to 
minimizing  errors  and  catching  them  early  due  to  the  complexity  of  the  system  being 
modeled  and  the  tune  constraints  associated  with  this  study.  The  model  was  built  one  sub 
model  and  gate  at  a  time  similar  to  Ysebaert’s  (2011)  simulation  research  on  F-22  Low 
Observable  panels. 
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The  first  gate  is  built  incrementally  and  animation  is  used  to  watch  the  basic 
processes  such  as  separating  and  batching  the  aircraft,  matching,  and  the  re- loop  function. 
Each  tune  a  new  process  was  added  the  verification  was  recompleted.  For  instance, 
during  the  verification  of  the  Characteristics  sub  model  the  Awaitinglnput  function  was 
allowing  one  more  aircraft  than  the  WIP  constraint  value  to  enter  the  system.  The 
equation  symbol  had  to  be  changed  from<  to  <  to  constrain  WIP.  Additionally,  a  delay 
function  (AV_Delay_SEizeDecisionl)  was  added  in  the  AVI  Number  sub  model  and  its 
counterparts  in  the  different  specialty  paths  after  visually  finding  a  later  aircraft  passing 
an  aircraft  in  the  hold  function  of  the  sub  model.  This  error  violated  the  first  in  first  out 
principals  of  the  real  system. 

Perhaps  the  most  significant  error  caught  was  the  looping  function  error  noticed 
through  animation  after  creating  Gate  2.  The  researcher  duplicates  Gate  1  lor  Gate  2  but 
the  ContinueFlag  attribute  is  deleted  from  the  Gate2_Start  module.  This  error  causes  the 
entities  to  exit  Gate  2  after  one  flow  day  because  the  ContinueFlag  was  not  reassigned  a 
value  of  one.  Recall  that  the  VBA  code  assigns  a  value  of  zero  when  the  next  flow  day  is 
the  start  of  the  subsequent  gate.  The  re- loop  modules  (Continue  AV?,  Continue  AT?, 
etc.)  allowed  the  aircraft  to  exit  the  gate  because  of  this  error.  Several  other 
abnormalities  highlighted  themselves  and  were  corrected  using  visual  verification 
through  animation  but  not  all  processes  are  visible  for  verification  purposes. 

The  more  detailed  functions  and  processes  are  verified  using  the  variable  display 
function  in  ARENA.  The  function  allows  a  variable,  attribute,  or  any  calculated  value  to 
be  seen  in  a  display  box.  The  numerical  value  of  each  variable  is  verified  versus  the 
expected  value  using  a  “trace”  type  method  as  described  by  Law  (2007).  The  model 
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animation  is  slowed  to  a  point  that  advancing  from  one  event  to  the  next  can  be  followed. 
The  attribute  and  variable  values  are  checked  against  expected  values,  first  with  one 
aircraft  flowing  through  the  system  then  increasing  the  WIP  to  the  constraint  value.  The 
most  complicated  verification  during  this  process  was  the  ProcessTime  attribute.  The 
VBA  code  pulls  in  the  man-hour  requirements  for  the  current  flow  day  prior  to  starting 
maintenance.  In  order  to  verily  the  correct  data  was  being  pulled  the  ProcessTime 
attribute  was  compared  to  the  current  flow  day  requirement  as  each  event  occurred. 
Additionally,  the  ProcessTime  attribute  was  compared  to  the  man-hour  input  data  in  the 
Model  Input  Data.xls  spreadsheet  based  on  the  PlaneNum  assigned  to  the  aircraft  in  order 
to  verily  the  correct  plane  data  was  input.  Throughout  the  verification  process  outputs 
were  continually  validated  for  accuracy  but  final  validation  is  done  by  comparing  outputs 
against  historical  data  and  calculations  from  the  real  system 

Validation 

In  model  validation,  the  researcher  seeks  to  ensure  the  model  has  sufficient 
accuracy  to  represent  the  real  system  (baseline  model  ~  real  system)  so  that  experimental 
comparisons  and  analysis  can  be  completed  on  modified  versions  of  the  system  (Carson, 
2002).  The  functions  of  the  model,  inputs,  and  expected  outputs  based  on  input 
adjustments  were  validated  throughout  the  model  building  and  verification  process.  The 
next  few  paragraphs  discuss  the  warm-up,  run  length,  and  replication  number  decision  as 
well  as  the  final  model  comparison  against  historical  data  from  the  real  system. 

The  warm-up  period  is  set  based  on  the  steady  parameter  of  having  the  current 
state  number  of  WIP  aircraft  in  the  system  at  all  times.  The  inter-arrival  time  and  warm- 
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up  period  are  calculated  using  the  target  throughput  of  aircraft  per  year  for  the  current 
state  and  aircraft  per  year  for  the  future  state  with  225  workdays  per  year.  To  find  the 
inter-arrival  time  for  the  system,  the  researcher  divides  workdays  by  target  throughput  for 
the  current  and  future  state. 

The  warm-up  period  for  the  model  is  1 00  days  for  the  current  state  in  order  to  get 
the  current  state  work- in-process  (WIP)  aircraft  in  the  system.  The  future-state  warm-up 
period  is  200  days  to  allow  for  future  state  WIP  aircraft  to  be  in  the  system  at  the  start  of 
statistic  collection  and  to  allow  enough  time  for  the  first  few  aircraft  to  exit  the  system. 
The  reasoning  for  allowing  the  first  aircraft  to  exit  the  system  is  that  personnel  resource 
constraints  do  not  affect  the  WIP  aircraft  until  many  more  aircraft  are  in  the  system  due 
to  the  doubling  ofpersonnel,  allowing  the  early  aircraft  to  flow  through  the  system  at  an 
unrealistic  rate  with  seemingly  minimal  resource  constraints.  The  warm-up  period  is 
added  to  the  run  length  decided  upon  in  the  next  step. 

The  coefficient  of  variation  (CV)  is  used  to  aid  in  the  decision  on  run  length  and 
replication  number.  The  current  state  CV  comparison  provides  a  clear  picture  on  the 
dispersion  characteristics  across  differing  model  run  lengths  and  replication  numbers. 
Figure  17  shows  that  a  small  amount  of  variation  exists  within  the  model  with  C  Vs 
between  .009  and  .022.  The  highest  CV  associated  with  a  3-year  run  length  is  reasonable 
because  less  than  28  aircraft  flow  through  each  of  the  3-year  replications  and  the 
variability  in  the  model  is  solely  dependent  on  the  type  and  number  of  aircraft  that  arrive. 
Conversely,  it  shows  that  a  run  length  of  8  to  10  years  is  favorable  in  order  to  increase  the 
statistical  accuracy  of  output  comparisons.  10  years  is  chosen  over  8  years  because  little 
CV  difference  exists  between  the  two  and  the  math  is  easier  for  comparing  experiment 
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results  against  the  5-year  multi- trade  (multi-skill)  analysis  completed  by  Ogden  Air 
Logistics  Center  (2012). 


5  10  15  20  25  30 


3  Years  s/p. 
^—5  Years  s/\i 
—  8  Years  s/p 
-^—10  Years  s/p 


Number  of  Replications 


Figure  17:  Run  Length  &  Rep  Comparison 


The  replication  number  selection  includes  reviewing  Figure  17  to  find  the  point 
where  the  CV  stabilizes  and  to  calculate  the  number  of  runs  needed  to  estimate  6  by  Y  to 
within  ±  one  day  witha  probability  of  95%  or  ana  levelof  .05  (Banks  et.  al,  2010,  p. 

43 1).  In  order  to  get  the  desired  level  of  precision,  Equation  10  is  applied  with  a  a  and  p 
value  for  the  8-year  and  10-year  run  lengths.  The  R  (replication)  number  needs  to  be 
greater  than  the  value  calculated.  The  results  in  Table  5  combined  withFigure  17  support 
the  decision  to  use  15  replications  for  the  10-year  run  length.  Table  5  also  includes  the 
calculated  R  values  for  two  days  of  precision,  but  the  higher  number  of  replications  are 
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chosen  for  a  precision  of  one  day  in  order  to  facilitate  statistical  accuracy  in  the  analysis 


portion  of  this  research. 


R> 


-,R- 1 
2 


(8) 


Table  5:  Replication  Calculation  for  Precision 


8-Year  Run  Length 

10-Year  Run  Length 

T  for  95%  Cl 

S  (10  Reps) 

£ 

>  R  Value 

T  for  95%  Cl 

S  (10  Reps) 

£ 

>  R  Value 

2.26 

2.54 

1 

33.02 

2.26 

2.04 

1 

21.23 

T  for  95%  Cl 

S  (15  Reps) 

£ 

>  R  Value 

T  for  95%  Cl 

S  (15  Reps) 

£ 

>  R  Value 

2.14 

2.2 

1 

22.26 

2.14 

1.77 

1 

14.43 

T  for  95%  Cl 

S  (20  Reps) 

£ 

>  R  Value 

T  for  95%  Cl 

S  (20  Reps) 

£ 

>  R  Value 

2.09 

2.26 

1 

22.38 

2.09 

1.92 

1 

16.17 

8-Year  Run  Length 

10-Year  Run  Length 

T  for  95%  Cl 

S  (10  Reps) 

£ 

>  R  Value 

T  for  95%  Cl 

S  (10  Reps) 

£ 

>  R  Value 

2.26 

2.54 

2 

8.25 

2.26 

2.04 

2 

5.31 

T  for  95%  Cl 

S  (15  Reps) 

£ 

>  R  Value 

T  for  95%  Cl 

S  (15  Reps) 

£ 

>  R  Value 

2.14 

2.2 

2 

5.57 

2.14 

1.77 

2 

3.61 

T  for  95%  Cl 

S  (20  Reps) 

£ 

>  R  Value 

T  for  95%  Cl 

S  (20  Reps) 

£ 

>  R  Value 

2.09 

2.26 

2 

5.59 

2.09 

1.92 

2 

4.04 

The  final  portion  of  validation  consists  o  ('comparing  flow  day  values  calculated 
for  the  real  system  with  the  flow  day  outputs  from  the  simulation  model.  The  researcher 
starts  by  finding  the  expected  flow  days  for  an  aircraft  based  on  the  man-hour 
requirements  of  the  aircraft.  F-22  Planners  provide  expected  flow  days.  The  burn  rate 
calculations  used  for  the  expected  flow  day  values  represent  a  linear  function  based  on 
the  number  of  man-hours  required  by  a  given  aircraft.  Through  observing  the  system  and 
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analyzing  the  data,  there  are  tunes  when  the  function  is  not  linear  due  to  cure  time  or 
constraints  on  the  number  of  personnel  able  to  work  an  aircraft  but  the  linear  burn  rate 
calculations  represent  the  best  comparison  values  available. 

Second,  the  researcher  calculates  the  average  man-hour  requirements  expected  for 
n  aircraft  flowing  through  the  system  based  on  the  input  percentages  discussed  in  the 
Inputs  section  of  this  chapter  and  shown  in  Table  3.  Third,  the  expected  flow  days  for 
each  of  the  four  types  of  aircraft  and  the  average  aircraft  are  calculated.  F  inally,  the 
expected  flow  days  are  compared  to  the  model  outputs  for  the  current  and  future  states 
depicted  in  Table  6  and  Table  7. 
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Table  6:  Current  State  Validation  -  Expected  vs.  Model  Flow  Days 


Current  State  Validation:  Expected  vs  Model  Flow  Days 

Aircraft  Category 

Man-Hour 

Requirement 

Expected 
Non-OT  Days 

Model 

Output 

Delta 

%  Delta 

Plane  1 

20471.60 

144.60 

141.92 

-2.68 

-1.86% 

Plane  2 

20127.10 

142.17 

141.69 

-0.48 

-0.34% 

Plane  3 

19900.60 

140.57 

138.80 

-1.77 

-1.26% 

Plane  4 

15663.10 

110.64 

121.69 

11.05 

9.99% 

Avg  Aircraft 

18617.06 

131.50 

133.99 

2.49 

1.89% 

Average  Aircraft  Flow  Days 

95%  Confidence  Interval 

Lower 

95%  Cl 

133.01 

Upper 

95%  Cl 

134.97 

The  Current  State  comparison  shows  the  average  flow  days  to  be  approximately 
2.5  days  over  expected  flow  days,  reflecting  a  longer  flow  time  than  expected  by  1 .89 
percent.  Additionally,  the  comparison  reveals  that  most  of  the  deviation  with  the  model 
and  expected  days  is  attributed  to  Plane  4,  the  highest  percentage  of  aircraft  flowing 
through  the  system.  The  same  comparison  is  conducted  on  the  Future  State  model  in 
Table  7. 


Table  7:  Future  State  Validation  -  Expected  vs  Model  Flow  Days 


Future  State  Validation:  Expected  vs  Model  Flow  Days 

Aircraft  Category 

Man-Hour 

Requirement 

Expected 
Non-OT  Days 

Model 

Output 

Delta 

%  Delta 

Plane  1 

20471.60 

144.60 

127.77 

-16.83 

-11.64% 

Plane  2 

20127.10 

142.17 

127.42 

-14.75 

-10.38% 

Plane  3 

19900.60 

140.57 

123.81 

-16.76 

-11.92% 

Plane  4 

15663.10 

110.64 

109.67 

-0.97 

-0.88% 

Avg  Aircraft 

19384.41 

136.92 

123.33 

-13.59 

-9.93% 

Average  Aircraft  Flow  Days 

95%  Confidence  Interval 

Lower 

95%  Cl 

122.75 

Upper 

95%  Cl 

123.91 

76 


The  Future  State  comparison  shows  the  average  flow  days  to  be  appro ximately 
13.59  days  below  expected  flow  days,  reflecting  a  shorter  flow  tune  than  expected  by 
9.93  percent.  The  123.33  average  flow  days  per  aircraft  is  even  below  the  flow  day  target 
with  overtime  of  125.61  days.  This  comparison  uses  the  same  burn  rate  from  the  Current 
State  comparison,  reflecting  the  current  method  for  forecasting  current  and  future  flow 
day  targets. 

Further  calibration  of  the  number  of  direct  labor  hours  per  day  available  by 
increasing  or  decreasing  model  day  length  from  20  hours  has  a  positive  relationship  to 
increasing  and  decreasing  model  flow  day  outputs  but  may  allow  for  more  or  less  direct 
labor  hours  during  experimentation  than  exists  in  the  real  system.  For  this  reason,  no 
additional  calibration  of  the  model  is  completed  in  order  to  be  more  aligned  to  the  6 1 .7 
percent  direct  labor  rate  (38.3  percent  indirect  labor  rate)  or  approximately  5.6  hour 
Output  Per  Man  Day  (OPMD)  per  technician  target  of  the  real  system  (Ogden  Air 
Logistics  Center,  2012). 

Discussions  withF-22  Heavy  Maintenance  Modification  Program  leadership  and 
SMEs  lead  the  researcher  to  the  conclusion  that  the  model  is  valid  for  the  purpose  of 
experimentation  and  decision  analysis.  I  reiterate  Carson’s  (2002)  quote  of  the  lamous 
statistician  George  Box,  “All  models  are  wrong.  Some  are  useful.”  (p.  53).  Throughout 
verification  and  validation,  the  model  has  proven  robust  in  producing  reasonable  values 
as  different  inputs  are  altered.  With  this  in  mind,  the  research  now  moves  to  the 
experimental  design  phase. 
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Marginal  Analysis 


A  marginal  analysis  method  is  used  for  analysis  in  this  research.  The  marginal 
analysis  design  includes  a  resource  add  experiment  in  the  current  and  future  state. 

Another  experiment  using  marginal  analysis  principles  considers  multi- skill  policies, 
multi-skilling  each  specialty  into  the  AP  resource,  and  to  concurrently  multi-skill  AP  into 
the  other  five  critical  path  specialties  in  the  F-22  Heavy  Maintenance  and  Modification 
Program  This  experiment  is  conducted  as  a  comparison  between  results  from  this 
research  and  the  five-year  BC  A  completed  by  Ogden  ALC  in  2012. 

The  first  experiment  is  to  add  one  person  by  specialty  for  each  run,  reflecting  the 
addition  of  three  additional  personnel  per  day  or  one  person  in  each  of  the  three  balanced 
shifts  (1800  labor  hours/person/year).  Each  experimental  level  contains  six  runs,  one  run 
for  each  specialty.  Note  that  only  one  specialty  has  added  personnel  from  the  baseline  for 
each  rua  The  specialty  that  shows  the  most  improvement  in  flow  days  from  the 
additional  personnel  using  the  Select- the- Best  Procedure  spreadsheet  developed  by  Banks 
et  aL  (2010)  is  retained  as  the  new  baseline  for  the  next  comparison.  This  method  of 
marginal  analysis  is  similar  to  Ysebaert’s  (2011)  “shopping  list”  method  but  uses  the 
Select- the- Best  Procedure  to  choose  the  best  system.  This  experiment  will  show  the 
impact  of  adding  personnel  on  flow  days  and  provide  a  comparison  for  multi-skilling 
improvements. 

In  the  event  that  a  system  is  not  differentiable  from  the  baseline  in  the  personnel 
add  experiments,  the  baseline  will  remain  the  same  and  additional  personnel  will  be 
added  in  each  specialty  and  compared  to  the  original  baseline.  The  process  is  repeated 
until  the  outputs  are  differentiable.  Each  experiment  ends  when  no  difference  exists  as 
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personnel  are  increased  or  when  the  expected  flow  days  with  overtime  are  met. 
Additionally,  Paired-t  test,  ANOVA,  and  Tukey- Kramer  multiple  comparison  analysis 
techniques  are  provided  for  informational  purposes  only  to  compare  against  the  results  of 
the  Select- the- Best  Procedure. 

The  second  set  of  experiments  for  the  current  state  of  the  model  is  to  multi-skill 
all  non-AP  (LO)  specialties  into  AP  and  all  AP  personnel  into  the  other  specialties.  The 
number  of  AP  personnel  multi-skilled  into  each  specialty  will  be  limited  to  no  more  than 
the  current  number  ofpersonnel  in  the  specialty.  The  experiment  will  include  10  levels, 
starting  with  allpersonnel  multi-skilled  and  reducing  the  multi- skilled  personnel  across 
the  board  by  10  percent  for  each  experimental  level  until  only  10  percent  of  employees 
are  multi-skilled.  This  experiment  provides  a  direct  comparison  to  the  multi- trade  policy 
analyzed  in  the  BCA  completed  by  Ogden  ALC  in2012  to  multi- trade  almost  all 
personnel  into  AP  and  all  AP  personnel  into  other  specialties.  Following  the  experiment 
to  multi-skill  only  into  and  out  of  AP,  a  targeted  approach  to  multi-skilling  is  conducted 
to  see  if  the  flow  day  targets  can  be  reached  using  different  multi-skilling  approachs. 

Similar  experiments  are  conducted  a  second  time  on  the  future  state  of  the  model. 
The  future  state  model  will  reflect  the  addition  of  the  Palmdale  work  and  will  have  an 
increased  aircraft  WIP  and  throughput  per  year.  Additionally,  the  future  state  input 
probabilities  for  the  type  of  aircraft  to  arrive  in  the  system  are  used  to  reflect  an  increase 
in  the  man-hour  requirements  associated  with  the  aircraft  that  previously  underwent 
maintenance  at  Palmdale. 

The  future  state  modelproduces  flow  day  averages  13.59  days  below  overtime 
targeted  flow  days  and  indicates  the  future  state  benefits  from  increasing  returns  of  scale 
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or  economies  ofscale  (de  NeufVille,  1990)  with  the  additional  personnel  and  workload. 
For  this  reason,  the  future  state  analysis  includes  reducing  personnel  by  5  percent  for 
each  level  down  to  a  25  percent  reduction.  The  level  that  closely  reflects  the  flow  day 
target  without  overtime  is  then  selected  as  a  baseline  for  multi-skilling  experiments. 

Targeted  multi-skilling  experiments  are  then  conducted  using  the  selected 
reduction  in  workforce  as  the  baseline  model.  The  multi-skilling  experiments  identify  if 
a  more  flexible  workforce  allows  for  reductions  in  personnel  requirements  for  the  future 
state  of  the  system.  Following  selection  of  the  best  targeted  multi-skilling  alternative, 
another  Personnel  Add  Marginal  Analysis  is  conducted  on  the  best  multi- skilled 
alternative.  This  experiment  identifies  the  number  of  personnel  required  to  meet  future 
state  requirements  with  a  multi-skilled  workforce.  Developing  the  model,  deciding  on 
inputs,  and  designing  experimentation  methods  leads  to  conducting  the  experiments  and 
analyzing  the  results. 
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IV.  Results  and  Analysis 


Chapter  Ove  rvie  w 

The  following  sections  discuss  employment  of  the  analysis  methods  and  the 
subsequent  results.  The  chapter  is  organized  in  two  sections  including  current  state 
analysis  and  future  state  analysis.  The  first  analysis  is  on  the  current  state  and  begins 
with  the  Personnel  Add  Marginal  Analysis. 

Current  State 

Personnel  Add  Marginal  Analysis 

In  the  Current  State  Personnel  Add  marginal  analysis,  one  additional  maintenance 
technician  is  added  within  each  of  the  six  specialty  and  each  add  is  treated  as  a  separate 
ran  (scenario).  The  Select-the-Best  Procedure  spreadsheet  provided  by  Banks  et  al. 
(2010)  is  used  to  select  the  run  with  the  best  improvement  over  the  baseline  using  a  95 
percent  confidence  level,  an  indifference  levelof  two  days,  and  a  sample  size  of  15 
replications.  The  selected  run  is  treated  as  the  new  baseline  model  and  the  subsequent 
personnel  adds  are  done  in  the  new  baseline  model  for  each  level  The  experiment 
mimics  adding  overtime  to  the  model  and  the  results  represent  the  number  of  overtime 
hours  needed  to  reach  the  average  expected  flow  day  target  of  120.64  days  with 
overtime-hourly  burn  rates  per  day  fromF-22  Planners.  The  expected  30,658  additional 
hours  of  overtime  shown  in  the  bottom  right  of  the  Table  8  is  based  on  real  world 
overtime  usage  for  the  aircraft  selected  as  inputs. 

The  analysis  runs  over  nine  levels  with  the  results  shown  in  Table  8.  The  bolded 
and  highlighted  runs  are  selections  as  the  best  system  for  each  level.  Additionally,  the 
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table  includes  P- values  for  each  run’s  Student-t  test  against  the  baseline  for  comparison 
with  the  Select-the-Best  Procedure  results.  The  Oneway  Analysis  of  flow  days  and 
Tukey-Kramer  HSD  multiple  comparison  results  fromJMP®,  included  in  Appendix  C, 
support  the  ranking  results  of  the  Select-the-Best  Procedure  even  though  statistically 
significant  differences  at  the  95  percent  confidence  level  do  not  exist  in  three  of  the  eight 
experimental  levels  in  which  selections  are  made. 


Table  8:  Current  State  -  Personnel  Add 


Current  State  -  Marginal  Analysis  (Personnel  Add)  Experiment 

Level 

Baseline  Flow  Days 
(133.99  Days) 

Data  Category 

AP 

AS 

AR 

AG 

AT 

AV 

Man-Hours 

Added/Year 

A1 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

1 

i 

1 

1 

1 

5,400 

Avg  Flow  Days 

134.10 

132.45 

133.53 

129.82 

132.88 

132.79 

Overtime  Cost 

3 

P  Value  (a  =  .05) 

0.65 

0.02 

0.5 

<.0001 

0.11 

0.08 

$218,430 

A2 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

1 

1 

2 

1 

1 

10,800 

Avg  Flow  Days 

129.94 

129.31 

130.17 

127.41 

127.92 

129.29 

Overtime  Cost 

6 

P  Value  (a  =  .05) 

0.87 

0.05 

0.65 

0.0025 

0.01 

0.49 

$436,860 

A3 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

1 

1 

3 

1 

1 

16,200 

Avg  Flow  Days 

127.13 

126.72 

126.7 

125.67 

125.26 

126.64 

Overtime  Cost 

9 

P  Value  (a  =  .05) 

0.7 

0.36 

0.34 

0.02 

0.005 

0.24 

$655,290 

A4 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

1 

1 

3 

2 

1 

21,600 

Avg  Flow  Days 

125.19 

125.26 

124.84 

123.54 

125.58 

124.4 

Overtime  Cost 

12 

P  Value  (a  =  .05) 

0.93 

0.99 

0.6 

0.03 

0.69 

0.28 

$873,720 

A5 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

1 

1 

4 

2 

1 

27,000 

Avg  Flow  Days 

124.13 

123.24 

123.18 

124.16 

123.46 

122.34 

Overtime  Cost 

15 

P  Value  (a  =  .05) 

0.48 

0.71 

0.66 

0.46 

0.92 

0.15 

$1,092,150 

A6 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

1 

1 

4 

2 

2 

32,400 

Avg  Flow  Days 

122.33 

121.28 

121.02 

122.04 

120.96 

122.14 

Overtime  Cost 

18 

P  Value  (a  =  .05) 

0.98 

0.13 

0.06 

0.66 

0.05 

0.77 

$1,310,580 

A7 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

1 

1 

4 

3 

2 

37,800 

Avg  Flow  Days 

121.34 

121 

120.59 

121.14 

121.4 

121.69 

Overtime  Cost 

21 

P  Value  (a  =  .05) 

0.6 

0.95 

0.61 

0.8 

0.55 

0.33 

$1,529,010 

A8 

Total  Technicians 

Added 

Additional  Techs/Shift 

3 

3 

3 

6 

5 

4 

48,600 

Avg  Flow  Days 

115.5 

119.4 

119.09 

120.01 

121.54 

121.42 

Overtime  Cost 

27 

P  Value  (a  =  .05) 

<.0001 

0.04 

0.016 

0.22 

0.44 

0.54 

$1,965,870 

A9 

Total  Technicians 

Added 

Additional  Techs/Shift 

5 

3 

3 

6 

5 

4 

59,400 

Avg  Flow  Days 

110.88 

111.89 

112.78 

112.02 

115.31 

115.95 

Overtime  Cost 

33 

P  Value  (a  =  .05) 

<.0001 

0.0003 

0.006 

0.0005 

0.84 

0.64 

$2,402,730 

Flow  Day  Target  with  Overtime 

120.64  Days 

OT  Hrs  Expected  in  Real  System 

30,658 

$1,240,116 

As  expected,  more  flow  day  reductions  occur  in  the  first  six  levels  of  adding 
technicians  with  diminishing  returns  with  each  additional  level.  Adding  AG  technicians 
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in  Levels  A1  and  A 2  reduces  the  average  flow  days  of  an  aircraft  by  approximately  seven 
days.  Adding  AT  personnel  in  Level  A3  reduces  average  flow  days  by  two  days.  Levels 
A3  through  A6  result  in  a  one  or  two-day  reduction  in  flow  days  for  each  level.  Note  that 
no  selection  is  made  in  Level  A7,  showing  no  decrease  in  flow  days  with  the  addition  of 
5,400  additional  labor  hours  per  specialty  per  year.  In  order  to  compensate  for 
diminishing  returns,  Levels  A8  and  A9  adds  two  personnel  in  each  specialty,  reflecting 
an  additional  six  total  personnel  per  level.  Level  A8  results  in  a  six  flow  day  reduction 
from  the  Level  A6  baseline,  indicating  that  large  amounts  of  AP  overtime  are  needed  in 
order  to  reduce  flow  days.  The  experiment  concludes  at  Level  A9  because  the  flow  day 
target  of  120.64  days  is  considered  met  in  Level  6  and  the  additional  10-day  reduction  in 
flow  days  from  Level  A6  to  Level  A9  almost  doubles  overtime  (additional personnel) 
requirements. 

As  indicated  in  Table  8,  32,400  overtime  hours  are  required  to  reach  the  120- flow 
day  target  in  Level  A6.  Using  the  rates  from  the  2012  BC  A  completed  by  Ogden  ALC, 
the  overtime  hours  equate  to  $1,310,580  in  overtime  costs  with  an  overtime  rate  of 
$40.45.  The  added  32,400  overtime  hours  closely  mirrors  the  expected  30,658  additional 
hours  of  overtime  based  on  real  world  overtime  usage  for  the  aircraft  selected  as  inputs. 
The  reduction  of  an  additional  six  flow  days  from  120  days  in  Level  A6  to  110  days  in 
Level  A9  requires  an  additional  27,000  hours  of  overtime,  costing  an  additional 
$1,092,150  in  overtime  costs. 

A  significant  finding  is  that  AP  (LO)  personnel  provide  the  best  return  only 
twice  in  the  9  levels  even  though  they  are  currently  viewed  as  the  main  bottleneck 
resource.  AG  and  AT  personnel  reflect  a  much  lower  portion  of  the  man-hour 
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requirements  and  provide  more  return  for  hours  added  in  5  of  the  9  levels.  Oneway 
analysis  of  total  flow  days  and  a  comparison  of  days  spent  in  each  gate,  shown  in  Table 
9,  reveals  that  Gates  1,  3,  and  Gate  7  see  the  most  flow  day  reductions  with  additional 
personnel  and  account  for  most  of  the  reductions  towards  meeting  flow  day  targets.  This 
finding  is  consistent  with  the  experimental  selections  to  add  additional  personnel  in  non- 
LO  specialties  for  5  of  the  9  experimental  levels  and  reflecting  that  a  majority  of  the  non- 
LO  man-hour  requirements  occur  outside  of  Gates  5  and  6.  Experimental  Level  A6  is 
highlighted  to  indicate  the  level  that  the  average  flow  day  target  is  met. 


Table  9:  Current  State  -  Personnel  Add  Gate  Analysis 


Experimental 

Level 

Personnel  Add  -  Days  Per  Gate  Comparison 

Personnel 

Added 

Gate  1 

Gate  2 

Gate  3 

Gate  4 

Gate  5 

Gate  6 

Gate  7 

Difference 

Baseline 

N/A 

23.63 

20.43 

23.11 

N/A 

14.82 

23.37 

16.49 

N/A 

A1 

3  AG 

-2.05 

-0.13 

-0.08 

N/A 

0.70 

0.49 

-2.90 

-3.96 

A2 

3  AG 

-2.80 

-0.48 

-0.40 

N/A 

1.29 

1.12 

-4.72 

-5.99 

A3 

3  AT 

-3.40 

-0.99 

-2.17 

N/A 

1.32 

1.39 

-4.53 

-8.38 

A4 

3  AG 

-4.17 

-1.33 

-1.91 

N/A 

1.49 

1.80 

-5.81 

-9.93 

A5 

3  AV 

-4.06 

-1.11 

-2.57 

N/A 

-2.77 

1.54 

-6.46 

-15.43 

A6 

3  AT 

-4.72 

-1.53 

-3.06 

N/A 

1.77 

1.53 

-6.57 

-12.58 

A7 

3 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

A8 

6  AP 

-4.59 

-1.86 

-3.30 

N/A 

-0.22 

-1.68 

-6.38 

-18.03 

A9 

6  AP 

-4.87 

-2.63 

-4.55 

N/A 

-1.56 

-2.93 

-6.11 

-22.65 

Another  item  of  note  is  that  flow  day  increases  are  observed  in  LO  Gates  5  and  6 
as  personnel  are  added  into  AG,  AT,  and  AV.  Showing  that  even  though  maintenance 
requirements  are  considered  independent  from  one  specialty  to  the  next,  complex 
interactions  exist  between  the  gates,  aircraft  within  the  gates,  and  specialties  in  use.  This 
also  shows  the  bottleneck  shift  more  towards  AP  heavy  gates  as  personnel  capacity 
increases  to  speed  up  processing  in  the  other  gates.  Following  analysis  of  personnel  adds, 
the  multi-skilling  analysis  is  conducted. 
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All  AP  Multi-skill  Analysis 


The  multi-skill  analysis  begins  by  multi-skilling  the  five  non-AP  specialties  into 
AP  and  all  AP  personnel  into  the  other  five  specialties  using  the  methodology  previously 
discussed.  Each  subsequent  run  reduces  the  number  of  multi-skilled  personnel  by  10 
percent  from  100  percent  down  to  10  percent. 

Multi-skilling  100  percent  of  the  workforce  provided  the  best  results,  increasing 
average  flow  days  by  1  day.  Furthermore,  multi-skilled  workforce  percentages  from  90 
percent  to  10  percent  increases  flow  days  with  the  worst  system  performance  of 
approximately  89  additional  days  at  60  percent.  Table  10  depicts  the  full  results, 
providing  the  number  of  multi-skilled  personnel  by  specialty  at  each  level,  the  average 
flow  days,  the  delta  from  the  baseline  (status  quo)  average  flow  days,  the  number  of 
multi-skilled  hours  per  year,  and  the  additional  cost  if  multi-skilled  personnel  are  paid 
more  per  hour  using  multi- trade  policies  from  the  2012  Multi- Trade  BCA.  The  cost  data 
reflects  the  $2.03  difference  between  WG-10  and  WG-1 1  employee  hourly  wages 
multiplied  by  the  number  of  multi-skilled  employee  hours  per  year  using  cost  data  from 
the  2012  BCA  (Ogden  ALC,  2012). 
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Table  10:  Current  State  -  All  AP  Multi-skill  Analysis 


Current  State  -  Multi-skill  AP  ALL  Experiment  Analysis 


Level 

Baseline  Flow  Days 
(133.99  Days) 

Category 

AP 

AS 

AR 

AG 

AT 

AV 

Total 

Multi-skilled 

B1 

100%  Multiski  lied 

Multi-ski  1  led/Shift 

32 

18 

4 

7 

3 

2 

198 

Avg  Flow  Days 

135.12 

Delta 

1.13 

Multi-skill  Hrs 

400,950 

Multi-Trade  Cost 

$813,929 

B2 

90%  Multiski  lied 

Multi-skilled/Shift 

29 

16 

4 

6 

3 

2 

180 

Avg  Flow  Days 

188.10 

Delta 

54.11 

Multi-skill  Hrs 

364,500 

Multi-Trade  Cost 

$739,935 

B3 

80%  Multiski  lied 

Multi-skilled/Shift 

26 

14 

3 

6 

2 

2 

159 

Avg  Flow  Days 

201.48 

Delta 

-67.49 

Multi-skill  Hrs 

321,975 

Multi-Trade  Cost 

$653,609 

B4 

70%  Multiski  lied 

Multi-skilled/Shift 

22 

13 

3 

5 

2 

1 

138 

Avg  Flow  Days 

220.56 

Delta 

86.57 

Multi-skill  Hrs 

279,450 

Multi-Trade  Cost 

$567,284 

B5 

60%  Multiski  lied 

Multi-skilled/Shift 

19 

9 

2 

4 

2 

1 

111 

Avg  Flow  Days 

223.51 

Delta 

89.52 

Multi-skill  Hrs 

224,775 

Multi-Trade  Cost 

$456,293 

B6 

50%  Multiski  lied 

Multi-skilled/Shift 

17 

9 

2 

4 

2 

1 

105 

Avg  Flow  Days 

213.70 

Delta 

79.71 

Multi-skill  Hrs 

212,625 

Multi-Trade  Cost 

$431,629 

B7 

40%  Multiski  lied 

Multi-skilled/Shift 

16 

8 

2 

3 

1 

1 

93 

Avg  Flow  Days 

142.28 

Delta 

-8.29 

Multi-skill  Hrs 

188,325 

Multi-Trade  Cost 

$382,300 

B8 

30%  Multiski  lied 

Multi-skilled/Shift 

13 

2 

2 

1 

1 

1 

60 

Avg  Flow  Days 

189.09 

Delta 

55.10 

Multi-skill  Hrs 

121,500 

Multi-Trade  Cost 

$246,645 

B9 

20%  Multiski  lied 

Multi-skilled/Shift 

7 

2 

2 

1 

0 

0 

36 

Avg  Flow  Days 

175.76 

Delta 

41.77 

Multi-skill  Hrs 

72,900 

Multi-Trade  Cost 

$147,987 

BIO 

10%  Multiski  lied 

Multi-skilled/Shift 

3 

2 

0 

1 

0 

0 

18 

Avg  Flow  Days 

150.80 

Delta 

16.81 

Multi-skill  Hrs 

36,450 

Multi-Trade  Cost 

$73,994 

The  results  show  that  a  fully  multi-skilled  workforce,  specifically  into  and  out  of 


AP  (LO),  provides  an  average  flow  day  increase  of  1  day  per  aircraft  at  a  cost  of 
$813,929.  The  other  nine  levels  indicate  significant  flow  day  increases.  The  results  lead 
to  further  interrogation  of  output  data  to  gain  insight  into  the  areas  within  the  system  that 
show  the  most  significant  impacts  from  multi-skilling  all  specialties  into  and  out  of  AP. 

The  next  comparison  looks  at  the  differences  in  flow  days  within  each  gate  across 
multi-skill  percentage  runs.  Negative  values  are  the  desired  state,  indicating  a  reduction 
in  days  from  the  baseline.  The  results  in  Table  1 1  show  that  most  of  the  flow  day 
increases  occur  in  Gates  2,  3,  5,  and  6,  showing  an  opposite  result  from  the  Personnel 
Add  Marginal  Analysis.  Additionally,  Gates  1  and  7  benefit  from  multi-skilling  at  9  of 
the  10  levels  but  negatively  impacting  Gates  5  and  6,  reflecting  similar  results  to  adding 
AG  personnel  in  the  Personnel  Add  experiment.  These  results  show  that  more  significant 
flow  day  increases  occur  in  Gates  5  and  6  (LO  gates)  in  which  approximately  50  percent 
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of  man-hour  requirements  on  each  aircraft  exist,  indicating  that  those  requirements  are 
delayed  when  AP  personnel  are  completing  maintenance  actions  in  the  five  other 
specialties.  Combined  with  the  interactional  effects  observed  in  the  personnel  add 
experiment,  this  shows  that  an  even  spread  of  AP  resources  into  the  other  five  specialties 
is  not  desirable  because  complex  interactions  exist  within  the  gates  and  periods  of  lower 
utilization  within  the  five  specialties  do  not  align  with  AP  requirements.  The  inverse  is 
also  true  for  lower  AP  utilization  periods  and  the  five  specialties’  requirements. 


Table  11:  Current  State  -  AP  ALL  Multi-skill  Gate  Analysis 


Level 

Mu 

ti-skill  All  AP  -  Days  Per  Gate  Comparison 

Multi-skill  % 

Gate  1 

Gate  2 

Gate  3 

Gate  4 

Gate  5 

Gate  6 

Gate  7 

Difference 

Baseline 

N/A 

23.63 

20.43 

23.11 

12.59 

14.82 

23.37 

16.49 

N/A 

B1 

100% 

7.55 

1.65 

-5.32 

-0.53 

2.22 

3.14 

-7.57 

1.14 

B2 

90% 

-3.21 

2.62 

0.94 

-0.56 

27.56 

33.96 

-7.62 

53.69 

B3 

80% 

-4.04 

5.91 

3.62 

-0.56 

31.15 

38.66 

-7.4 

67.34 

B4 

70% 

-4.33 

11.61 

7.01 

-0.57 

39.15 

43.09 

-8.21 

87.75 

B5 

60% 

-2.12 

12.26 

7.4 

-0.53 

34.94 

43.33 

-5.28 

90 

B6 

50% 

0.77 

9.19 

6.45 

-0.51 

31 

38.03 

-4.82 

80.11 

B7 

40% 

1.35 

0.47 

-3.52 

-0.4 

5.66 

7.55 

-2.98 

8.13 

B8 

30% 

-7.05 

3.56 

2.94 

-0.52 

29.38 

33.29 

-6.63 

54.97 

B9 

20% 

-3.05 

0.84 

1.57 

-0.19 

18.66 

23.52 

-0.73 

40.62 

BIO 

10% 

0.56 

0.01 

-0.61 

-0.1 

6.2 

9.1 

1.03 

16.19 

The  results  also  show  that  flow  day  increases  occur  in  non-LO  gates  as  multi¬ 
skilling  decreases,  indicating  that  combining  the  smaller  specialties  into  AP  (LO)  has  a 
negative  impact  in  Gates  2  and  3  at  the  80  to  50  percent  multi-skilled  levels  because  AP 
has  a  much  larger  man-hour  requirement  and  uses  the  multi- skilled  resource  more 
frequently.  The  unavailability  of  the  multi-skilled  person  to  complete  their  primary 
specialty  has  a  much  larger  impact  on  the  smaller  specialties  than  on  the  larger  resource 
pool  of  AP  as  multi-skilled  levels  decrease. 
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Another  consideration  is  model  Limitation  4,  in  which  a  multi-skilled  person 
seized  for  maintenance  actions  is  held  to  complete  the  task  even  though  the  primary 
specialty  becomes  available.  This  limitation  may  increase  flow  days  for  unfavorable 
policies  because  demand  for  the  multi- skilled  pairings  are  not  offset  and  both  resources 
have  peak  demands  concurrently.  This  explains  the  larger  increases  in  flow  days  with  the 
mid  and  lower  level  percentages  of  multi-skilled  personnel  and  highlights  the 
unlavorability  of  the  proposed  policy. 

These  insights  lead  the  researcher  to  conduct  a  targeted  multi-skilling  experiment 
to  see  if  flow  day  reductions  are  attained  through  other  multi-skilling  policies. 

Targeted  Multi-skill  Analysis 

The  targeted  analysis  uses  insights  from  the  AP  All  multi-skill  analysis  as  a 
starting  point.  The  Targeted  experiments  focus  on  a  paired  method  versus  multi-skilling 
a  few  specialties  into  each  other  by  using  insights  from  utilization  rates  and  man-hour 
requirements  within  each  gate  to  selectively  multi- skill  each  specialty  with  only  one  other 
specialty.  Additional  considerations  are  made  for  current  resource  capacity,  seeking  to 
pair  resource  pools  with  lower  utilization  rates  with  resource  pools  closest  in  size  that 
required  overtime  hours  in  the  Personnel  Add.  Table  12  shows  the  results  of  the  Targeted 
Multi-skilling  experiments. 


Table  12:  Current  State  -  Targeted  Multi-skill  Analysis 


Current  State  -  Targeted  Multi-skill  Experiment  Analysis 

Level 

Flow  Days  / 
Delta  w/Baseline 

Data  Category 

AP 

AS 

AR 

AG 

AT 

AV 

Total 

Multiskilled 

Cl 

116.29 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

198 

-17.7 

Multi-skilled/Shift 

32 

18 

4 

7 

3 

2 

$813,929 

C2 

117.55 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

150 

-16.44 

Multi-skilled/Shift 

16 

18 

4 

7 

3 

2 

$616,613 

C3 

121.3 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

126 

-12.69 

Multi-skilled/Shift 

8 

18 

4 

7 

3 

2 

$517,955 

C4 

124.66 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

114 

-9.33 

Multi-skilled/Shift 

4 

18 

4 

7 

3 

2 

$468,626 

C5 

123.01 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

102 

-10.98 

Multi-skilled/Shift 

0 

18 

4 

7 

3 

2 

$419,297 

C6 

129.15 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

96 

-4.84 

Multi-skilled/Shift 

8 

8 

4 

7 

3 

2 

$394,632 

C7 

119.7 

Additional  Specialty 

AS 

AP 

AT 

AV 

AR 

AG 

150 

-14.29 

Multi-skilled/Shift 

16 

18 

4 

7 

3 

2 

$616,613 

C8 

121.5 

Additional  Specialty 

AS 

AP 

AV 

AT 

AG 

AR 

150 

-12.49 

Multi-skilled/Shift 

16 

18 

4 

7 

3 

2 

$616,613 

C9 

134.57 

Additional  Specialty 

AR 

AG 

AP 

AS 

AV 

AT 

114 

0.58 

Multi-skilled/Shift 

4 

18 

4 

7 

3 

2 

$468,626 

Flow  Day  Target  with  Overtime 

120.64  Days 

OT  Hours  Expected  in  Real  System 

30,658 

$1,240,116 

The  initial  pairings  include  AP  and  AS,  AG  and  AR,  and  AT  with  AV.  The  first 
experimental  run  reduces  flow  day  averages  by  approximately  17.7  days  or  13.2%  from 
the  baseline  of  133.99  days.  The  second  run  decreases  the  number  of  AP  personnel 
multi-skilled  by  50  percent  to  16  personnel  per  shift,  resulting  in  a  flow  day  reduction  of 
16.4  days  or  12.2%  from  the  baseline.  The  following  seven  runs  consider  other  levels  of 
multi-skilled  personnel  and  pairings  but  the  first  two  initial  pairings  resulted  in  the  largest 
flow  day  reductions,  producing  results  below  the  flow  day  target  with  overtime  of  120.64 
days. 

A  Oneway  analysis,  Student’s  t  and  Tukey-Kramer  means  comparison  is 
conducted  on  the  flow  day  outputs  to  identify  statistically  significant  differences.  Using 
the  Student’s  t  and  Tukey-Kramer  methods  with  ana  level  of  .05,  no  statistically 
significant  difference  exists  between  the  lowest  two  flow  day  averages  in  Levels 
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(Treatments)  C 1  and  C2.  The  Oneway  analysis  in  Figure  18 :  Current  State  -  Targeted 


Multi-skilling  JMP®  Oneway  Analysis,  depicts  overlapping  95  percent  confidence 
intervals  for  Treatment  1  and  2  as  well.  Therefore,  both  levels  are  retained  as  the  best  for 
future  analysis. 


Figure  18:  Current  State  -  Targeted  Multi-skilling  JMP®  Oneway  Analysis 


Overall,  the  Targeted  Multi-skilling  Analysis  shows  that  selective  multi-skilling 
of  198  or  150  personnel  in  paired  specialties  can  reduce  the  average  flow  days  ofan 
aircraft  by  17.7  and  16.44  days  with  no  additional  personnel  resources.  If  multi-trade 
type  pay  incentives  are  applied,  the  additional  cost  for  198  multi- skilled  employees  is 
$813,929  per  year  or  $616,613  for  150  personnel  using  the  2012  BCA  rates  (Ogden 
ALC).  With  the  Personnel  Add  and  Multi-skilling  experiments  complete,  results 
comparisons  are  conducted  to  identify  and  summarize  the  best  alternatives  for  the  current 
state  of  the  system. 
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Current  State  -  Results  Summary 


Comparison  of  the  current  state  results  compiles  results  data  from  Tables  8,  10, 
and  12.  It  also  includes  additional  model  run  outputs  for  the  Targeted  Multi-skilling 
lowest  flow  day  scenarios  from  Table  12,  in  which  the  efficiency  of  the  labor  force  is 
reduced  within  each  run  from  98  percent  down  to  90  percent.  The  efficiency  changes 
reflect  previous  research  indicating  varying  levels  ofproficiency  reductions  associated 
with  a  more  generalized  workforce  (Levien,  2010). 

Table  13  provides  a  list  of  the  best  results  from  each  experimental  design  as  well 
as  results  from  across  the  analyses  with  similar  flow  day  outputs.  The  delta  column 
indicates  the  difference  from  the  baseline  (status  quo)  model  outputs.  The  annual 
throughput  column  indicates  the  aircraft  throughput  per  year,  calculated  by  taking  the 
total  throughput  of  aircraft  for  the  10- year  run  length  and  dividing  by  10.  The  cost 
column  is  calculated  by  taking  the  overtime  and  multi- skill  hour  column  value  and 
multiplying  by  $40.45  per  overtime  hour  or  $2.03  per  multi- skill  hour  (Ogden  ALC, 
2012). 
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Table  13:  Current  State  -  Results  Summary 


Current  State  Experiments  -  Results  Summary 

Experiment  Name  (Level) 

Flow  Days 

Delta 

w/Baseline 

Annual 

Throughput 

Overtime  / 
Multi-skill  Hrs 

Annual 

Cost 

Baseline 

133.99 

N/A 

9.90 

N/A 

N/A 

Personnel  Add  -  Overtime  (A8) 

115.5 

-18.5 

11.30 

48,600 

$1,965,870 

Targeted  Multi-skilling  (Cl) 

116.29 

-17.7 

11.27 

400,950 

$813,929 

Targeted  Multi-skilling  (C2) 

117.55 

-16.4 

11.14 

303,750 

$616,613 

Targeted  98%  Efficient  (Cl) 

119.8 

-14.2 

11.03 

400,950 

$813,929 

Targeted  98%  Efficient  (C2) 

120.92 

-13.1 

10.87 

303,750 

$616,613 

Personnel  Add  -  Overtime  (A6) 

120.96 

-13.0 

10.92 

32,400 

$1,310,580 

Targeted  Multi-skilling  95%  Efficient  (Cl) 

123.16 

-10.8 

10.77 

400,950 

$813,929 

Personnel  Add  -  Overtime  (A4) 

123.54 

-10.5 

10.67 

21,600 

$873,720 

Targeted  Multi-skilling  95%  Efficient  (C2) 

124.9 

-9.1 

10.60 

303,750 

$616,613 

Personnel  Add  -  Overtime  (A3) 

125.26 

-8.7 

10.58 

16,200 

$655,290 

Personnel  Add  -  Overtime  (Al) 

129.82 

-4.2 

10.24 

5,400 

$218,430 

Targeted  Multi-skilling  90%  Efficient  (Cl) 

130.22 

-3.8 

10.20 

400,950 

$813,929 

Targeted  Multi-skilling  90%  Efficient  (C2) 

131.25 

-2.7 

10.15 

303,750 

$616,613 

AP  All  Multi-skill  100%  (Bl) 

135.12 

1.1 

9.70 

400,950 

$813,929 

Average  Flow  Day  Target  w/OT 

120.64 

Expected  OT  Cost  for  Real  System 

$1,240,116 

The  summarized  results  show  that  multi-skilling  a  workforce  using  Targeted 
policies  with  no  efficiency  losses  produce  flow  day  averages  similar  to  the  best  overtime 
scenario.  The  cost  difference  between  the  best  Targeted  policy  (C 1)  and  overtime  policy 
(A8)  shown  in  Table  13  is  approximately  $1,151,941  annually  or  $5,759,705  over  five 
years.  The  second  best  Targeted  policy  provides  a  two-day  difference  from  the  best 
overtime  result  with  a  cost  difference  of  $1,349,257  annually  or  $6,746,285  over  five 
years.  In  both  Targeted  Experiments  Cl  and  C2,  the  flow  day  outputs  fall  below  the  flow 
day  target  of  120.64  days  using  overtime  hours  per  day  burn-rates  from  the  Suggested 
Flow  Day  spreadsheet  provided  by  F-22  planners. 

Furthermore,  reductions  in  efficiency  levels  to  98  percent  (reflective  of  2  percent 
longer  processing  times)  for  the  Targeted  policies  still  fall  below  the  flow  day  target  with 
overtime  of  120.64  days.  Again,  reflecting  a  cost  difference  between  multi- skilling  and 
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overtime  of  $693, 967  annually  for  Targeted  Experiment  2  (C2)  and  $496,651  annually 
for  Targeted  Experiment  1  (Cl),  both  at  98  percent  efficiency.  Further  reducing  the 
efficiency  level  of  the  workforce  diminishes  the  cost  differences  between  overtime  and 
Targeted  multi-skilling  with  break-even  points  occurring  at  95  percent  efficiency  for  both 
Targeted  policies.  Indicating  that  maintenance  actions  can  take  5  percent  longer  before 
the  overtime  and  multi-skilling  policies  are  approximately  equivalent  in  terms  of  cost. 

One  point  of  interest  is  that  Targeted  Multi-skilling  Experiment  C2  includes  48  fewer 
multi-skilled  personnel  than  Experiment  Cl.  If  efficiency  losses  occur,  the  impacts 
would  be  less  under  the  policy  with  fewer  multi-skilled  personnel. 

Finally,  the  results  support  the  hypothesis  of  productivity  increases  and  cost 
savings  from  multi-skilling  proposed  in  the  BCA  completed  by  Ogden  ALC  in  2012. 
However,  the  results  indicate  that  multi-skilling  all  five  of  the  non-LO  FWS  maintenance 
specialties  into  LO,  as  prescribed  by  the  BCA,  is  not  a  desirable  policy  in  terms  of  flow 
day  reductions,  annual  aircraft  throughput,  and  cost.  Following  the  analysis  and 
comparison  of  overtime  and  multi-skilling  in  the  current  state  model,  the  researcher 
moves  to  experimentation  and  analysis  of  the  future  state  model. 

Future  State 

The  future  state  analysis  includes  three  sections,  a  Personnel  Reduction  Analysis, 
Targeted  Multi-skilling  Analysis,  and  Multi-skilled  Personnel  Add  Analysis.  The  goal  of 
this  group  of  experiments  is  to  ascertain  whether  targeted  multi-skilling  policies  allow  for 
a  reduction  in  future  maintenance  personnel  requirements.  The  first  experiments 
conducted  are  the  Personnel  Reduction  Analysis. 
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Personnel  Reduction  Analysis 


The  future  state  analysis  begins  with  double  the  personnel  resources  from  the 
current  state  and  reduces  personnel  by  5  percent  for  each  level  down  to  a  25  percent 
reduction.  Table  14:  Future  State  -  Personnel  Reduction  displays  the  results  from  these 
experiments.  The  flow  day  target  indicated  at  the  bottom  of  the  table  represents  the  target 
flow  days  for  the  average  aircraft  requiring  19,384  criticalpath  hours.  The  136.92  day 
target  is  calculated  using  non-overtime  burn-rate  calculations  from  the  Banded  Flow  Day 
Suggestions  provided  by  F-22  Planners.  The  average  aircraft  maintenance  labor- hour 
requirement  would  be  approximately  22,000  criticalpath  hours  without  adjustments  for 
over  forecasting  of  AP  hours,  reflecting  an  aircraft  requiring  over  25,000  total 
maintenance  hours  with  an  overtime  flow  day  target  of  144  days. 


Table  14:  Future  State  -  Personnel  Reduction 


Future  State  -  Personnel  Reduction  Experiment 

Level 

Redux  % 

Data  Category 

AP 

AS 

AR 

AG 

AT 

AV 

Personnel 

Per  Shift 

Total 

Personnel 

Cost 

Flow  Days 

Baseline 

N/A 

Personnel 

64 

36 

8 

14 

6 

4 

132 

396 

$27,449,037 

123.33 

D1 

0.05 

Reduction  /  Shift 

3 

2 

0 

1 

0 

0 

6 

18 

$1,247,684 

Total  /  Shift 

61 

34 

8 

13 

6 

4 

126 

378 

$26,201,354 

129.08 

D2 

0.1 

Reduction  /  Shift 

6 

4 

1 

1 

1 

0 

13 

39 

$2,703,314 

Total  /  Shift 

58 

32 

"  7 

13 

’  5 

4 

119 

357 

$24,745,723 

132.92 

D3 

0.15 

Reduction  /  Shift 

10 

5 

1 

2 

1 

1 

20 

60 

$4,158,945 

Total  /  Shift 

54 

31 

7 

12 

5 

3 

112 

336 

$23,290,092 

142.49 

D4 

0.2 

Reduction  /  Shift 

13 

7 

2 

3 

1 

1 

27 

81 

$5,614,576 

Total  /  Shift 

51 

29 

6 

11 

’  5 

3 

105 

315 

$21,834,461 

150.38 

D5 

0.25 

Reduction  /  Shift 

16 

9 

2 

4 

2 

1 

34 

102 

$7,070,207 

Total  /  Shift 

48 

27 

6 

10 

4 

3 

98 

294 

$20,378,831 

162.66 

Flow  Day  Target  w/o  OT 

136.92 

The  results  show  that  each  reduction  of  personnel  by  5  percent  increases  average 
flow  days  between  5.75  days  and  12  days  more  than  the  previous  reduction  level. 
Additionally,  a  cost  reduction/avoidance  of  more  than  $1,247,684  is  realized  for  each  5 
percent  reduction  in  personnel. 
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The  15  percent  reduction  level  is  chosen  over  the  10  percent  reduction  level  as  the 
baseline  for  multi-skilling  experiments  because  it  is  above  the  flow  day  target  of  136.92 
days.  The  researcher  seeks  to  start  above  the  target  flow  days  in  order  to  allow  for 
maximization  of  multi-skilling  benefits  associated  with  personnel  reductions.  Subsequent 
experiments  conducted  in  a  multi-skilled  environment  re- add  the  personnel  that  most 
benefit  the  system.  Following  the  identification  of  the  15  percent  reduction  level  as  the 
baseline  for  multi-skilling,  a  targeted  multi-skilling  experiment  is  completed  on  the  new 
baseline. 

Targeted  Multi-skill  Analysis 

The  future  state  Targeted  Multi-skill  Analysis  used  insights  garnered  from  the 
current  state  experiments  to  multi- skill  the  new  baseline  model,  reflecting  a  15  percent 
reduction  in  personnel  resources.  The  analysis  included  11  experimental  levels,  multi¬ 
skilling  different  numbers  of  personnel  and  specialties  at  each  level.  Table  15:  Future 
State  -  Multi-skill  Targeted  Analysis  (15%  Reduction)  shows  the  results  of  the  analysis. 

The  multi-skilled  cost  is  calculated  by  multiplying  the  total  multi-skilled  hours 
added  by  $2.03,  the  difference  between  WG-10  and  WG-1 1  burdened  hourly  rates.  The 
total  cost  is  garnered  by  multiplying  the  total  number  ofpersonnelby  the  WG-10  wage 
rate  of  $34.23  and  then  adding  the  multi- skilled  cost  for  the  number  of  multi- skilled 
employees.  Both  cost  columns  represent  the  dollar  figure  on  an  annual  basis.  The  flow 
day  targetwithovertime  is  calculated  using  the  overtime  burn-rates  from  the  Banded 
Flow  Day  Suggestions  provided  by  F-22  Planners  and  is  displayed  in  the  bottom  right  of 
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the  table.  Additionally,  the  baseline  flow  days  represent  the  15  percent  Personnel 
Reduction  scenario  value  of  142.49  flow  days. 


Table  15:  Future  State  -  Multi-skill  Targeted  Analysis  (15%  Reduction) 


Future  State  -  Targeted  Multi-skill  Experiment  Analysis  (15%  Personnel  Reduction) 

Level 

Flow  Days  / 
Delta  w/Baseline 

Data  Category 

AP 

AS 

AR 

AG 

AT 

AV 

Total 

Multiskilled  / 

Cost 

Total 

Personnel  / 

Cost 

El 

133.14 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

186 

336 

-9.35 

Multi-skilled/Shift 

22 

13 

7 

12 

5 

3 

$764,600 

$24,054,692 

E2 

135.52 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

162 

336 

-6.97 

Multi-skilled/Shift 

18 

9 

7 

12 

5 

3 

$665,942 

$23,956,034 

ES 

138.06 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

138 

336 

-4.43 

Multi-skilled/Shift 

14 

5 

7 

12 

5 

3 

$567,284 

$23,857,376 

E4 

147.24 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

114 

336 

4.75 

Multi-skilled/Shift 

10 

1 

7 

12 

5 

3 

$468,626 

$23,758,718 

E5 

135.25 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

153 

336 

-7.24 

Multi-skilled/Shift 

22 

13 

3 

5 

5 

3 

$628,945 

$23,919,037 

ES 

127.22 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

336 

336 

-15.27 

Multi-skilled/Shift 

54 

31 

7 

12 

5 

3 

'  $1,381,212 

$24,671,304 

E7 

128.32 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

288 

336 

-14.17 

Multi-skilled/Shift 

38 

31 

7 

12 

5 

3 

’  $1,183,896 

$24,473,988 

E8 

131.84 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

240 

336 

-10.65 

Multi-skilled/Shift 

38 

15 

7 

12 

5 

3 

$986,580 

$24,276,672 

E9 

128.76 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

240 

336 

-13.73 

Multi-skilled/Shift 

22 

31 

7 

12 

5 

3 

$986,580 

$24,276,672 

E10 

130.05 

Additional  Specialty 

AS 

AP 

AT 

AV 

AR 

AG 

288 

336 

-12.44 

Multi-skilled/Shift 

38 

31 

7 

12 

5 

3 

'  $1,183,896 

$24,473,988 

Ell 

132.23 

Additional  Specialty 

AS 

AP 

AG 

AR 

AV 

AT 

192 

336 

-10.26 

Multi-skilled/Shift 

22 

15 

7 

12 

5 

3 

$789,264 

$24,079,356 

Baseline  Flow  Days  from  15%  Personnel  Reduction 

142.49 

Flow  Day  Target  with  Overtime 

125.61 

The  experiments  paired  the  smaller  specialties  for  multi-skilling  and  paired  AP 
(LO)  and  AS  (Sheet  Metal)  personnel  based  on  previous  experimental  results.  The 
results  show  that  targeted  multi-skilling  of  the  reduced  workforce  can  reduce  flow  days 
by  15.27  days  in  Level  E6,  within2  days  of  overtime  targets  at  an  annual  cost  of 
$24,671,304  with  all  personnel  multi-skilled.  Levels  E7  and  E9  show  similar  results  by 
reducing  flow  days  by  appro ximately  14  days  at  a  cost  of  $24,473,988  and  $24,276,262. 
The  difference  of  48  multi-skilled  AP  personnel  between  the  two  levels  indicates 
diminishing  returns  with  higher  levels  of  multi-skilled  AP  technicians  into  AS. 
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Furthermore,  all  experimental  levels  except  Level  E4  resulted  in  flow  day 
reductions.  Level  E4  multi-skilled  low  numbers  of  AP  and  AS  personnel  with  no 
changes  to  the  other  specialties  and  provided  the  least  desirable  flow  day  averages. 

Levels  El  through  E4  indicate  that  decreasing  AP  and  AS  multi- skilled  personnel  have 
an  inverse  affect  on  flow  day  averages. 

Level  E5  mimics  Level  E2  but  decreases  the  multi-skilled  personnel  in  the  AG 
and  AR  pairing,  resulting  in  a  2-day  increase  in  flow  days  from  Level  E2.  This 
comparison  supports  the  finding  that  multi-skilling  the  entirety  of  personnel  in  the 
smaller  specialties  provides  the  best  results.  Limitation  4  may  further  hinder  targeted 
optimization  of  the  smaller  specialties  because  it  causes  multi-skilled  personnel  to 
comp  letc  maintenance  actions  even  if  the  primary  specialty  becomes  available, 
overestimating  the  need  for  multi-skilled  personnel.  For  this  reason.  Levels  E6  through 
El  1  considered  only  multi-skilling  the  entirety  of  the  smaller  specialties. 

The  small  difference  in  flow  days  between  Level  E6,  E8,  and  E9  led  the 
researcher  to  complete  a  JMP®  Oneway  Analysis  of  flow  days  for  all  levels,  displayed  in 
Figure  19.  The  analysis  appears  to  show  overlap  of  the  95  percent  confidence  intervals  of 
the  Levels  E6,  E8,  and  E9.  Further  interrogation  is  conducted  through  means 
comparisons  for  all  levels  using  Tukey-Kramer  methods  in  JMP®.  The  results  are 
displayed  in  Figure  71  in  Appendix  C,  indicating  no  statistically  significant  difference 
exists  between  the  means  of  Levels  E6  and  E7  or  Levels  E7  and  E9  even  though  resource 
differences  of  48  multi-skilled  AP  personnel  exist  between  each  pair. 
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Figure  19:  Future  State  -  Targeted  Multi-skill  Oneway  Analysis 


In  order  to  capture  a  larger  picture  of  the  trade  space  between  multi-skilling 
numbers  for  AP,  Levels  E6  and  E9  are  retained  for  the  next  experiment,  the  Multi-skill 
Personnel  Add  Analysis. 


Multi-skilled  Personnel  Add  Analysis 

The  Multi-skilled  Personnel  Add  Analysis  is  conducted  identically  to  the  Current 
State  Personnel  Add.  One  person  is  added  per  run  (scenario)  for  each  specialty  and  the 
best  run  is  selected  using  the  Select- the- Best  Procedure  spreadsheet  provided  by  Banks  et 
aL  (2010).  This  analysis  includes  two  experiments,  one  using  Level  6  (E6)  resource 
values  fromthe  Targeted  Multi-skilling  experiments  as  a  baseline  and  one  using  Level  9 
(E9)  values.  The  first  experiment  completed  is  the  personnel  add  for  E6. 

The  E6  personnel  add  spans  four  levels,  adding  one  person  to  each  of  three  multi- 
skilled  specialties  and  selecting  the  best  alternative.  The  results  are  depicted  in  Table  16: 
Future  State  -  15%  Redux  Targeted  Multi-skilling  Add  (E6).  Since  all  personnel  are 
multi- skilled,  the  total  annual  cost  is  calculated  by  multiplying  the  total  personnel  by 
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2025  hours  per  year  (representative  of  the  225  day  per  year  planning  factor/model  run 
length  with  nine  hour  shifts),  then  multiplying  by  the  burdened  WG- 1 1  wage  rate  of 
$36.26  (Ogden  ALC,  2012).  The  shaded  results  represent  the  selection  at  each  level  and 
the  P  Value  represents  the  Student’s  t  comparison  against  the  baseline.  Student’s  t  and 
Tukey-Kramer  multiple  comparisons  from  JMP®  are  included  in  Appendix  C  for 
comparison  against  the  Select-The-Best  Procedure.  The  comparison  rankings  from  the 
baseline  agree  with  the  selections  made  for  each  experiment. 


Table  16:  Future  State  -  15%  Redux  Targeted  Multi-skilling  Add  (E6) 


Future  State  - 15  %  Reduction  Targeted  Multi-skilling  (E6)  (Personnel  Add) 

Level 

Multi-skilled 

Baseline 
(127.22  Days) 

Data  Category 

AP/AS 

AR/AG 

AT/AV 

Total 

Personnel  / 
Annual  Cost 

FI 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

1 

1 

339 

Avg  Flow  Days 

126.23 

125.54 

126.31 

Cost 

3 

P  Value  (a  =  .05) 

0.01 

<.0001 

0.02 

$24,891,584 

F2 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

2 

1 

342 

Avg  Flow  Days 

124.24 

124.32 

124.82 

Cost 

6 

P  Value 

<.0001 

0.0002 

0.02 

$25,111,863 

F3 

Total  Technicians 

Added 

Additional  Techs/Shift 

2 

2 

1 

345 

Avg  Flow  Days 

123.58 

123.16 

124.04 

Cost 

9 

P  Value 

0.02 

0.0005 

0.5 

$25,332,143 

F4 

Total  Technicians 

Added 

Additional  Techs/Shift 

2 

3 

1 

348 

Avg  Flow  Days 

122.41 

121.89 

122.45 

Cost 

12 

P  Value 

0.03 

0.0007 

0.04 

$25,552,422 

Baseline  Flow  Days  w/  No  Personnel  Reduction 

123.33 

Flow  Day  Target  w/  OT 

125.61 

The  flow  day  target  is  reached  in  the  first  level  with  the  addition  of  three  AR/AG 
personnel,  totaling  339  multi- skilled  personnel  with  a  total  annual  cost  of  $24,981,584. 
Note  that  Level  F3  reaches  the  baseline  future  state  (with  no  personnel  reduction)  model 
flow  day  average  of  123.33  with  345  personnel,  53  fewer  personnel  than  the  baseline  of 
398.  The  experiments  are  concluded  at  Level  F4  with  348  multi-skilled  personnel  and  a 
flow  day  average  of  121.89  days  at  an  annual  cost  of  $25,552,422.  Following  the  E6 
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Personnel  Add,  the  same  experiments  are  conducted  on  the  second  retained  level  from  the 
Targeted  Multi-skilling  Analysis,  E9. 

The  E9  personnel  add  includes  seven  experimental  levels  and  four  runs 
(scenarios)  per  level.  The  only  difference  between  the  E6  experiments  and  these 
experiments  are  the  AP  personnel  that  are  not  multi-skilled,  requiring  one  additional  run 
per  level.  The  results  for  the  seven  experimental  levels  with  selections  shaded  are 
displayed  in  Table  17:  Future  State  -  15%  Redux  Targeted  Multi-skilling  Add  (E9). 

The  Student’s  tand  Tukey- Kramer  multiple  comparisons  from  JMP®  are  included  in 
Appendix  C  for  comparison  against  the  Select- The- Best  Procedure  and  again  support  the 
selections  made. 


Table  17:  Future  State  -  15%  Redux  Targeted  Multi-skilling  Add  (E9) 


Future  State  - 15  %  Reduction  Targeted  Multi-skilling  (E9)  (Personnel  Add) 

Level 

Multi-skilled 

Baseline 
(128.76  Days) 

Data  Category 

AP 

AP/AS 

AR/AG 

AT/AV 

Total 

Multi-skilled  / 
Annual  Cost 

Total  Personnel  / 
Annual  Cost 

G1 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

1 

1 

1 

243 

339 

Avg  Flow  Days 

128.57 

128.47 

127.86 

128.72 

Multi-skilled  Cost 

Total  Cost 

3 

P  Value  (a  =  .05) 

0.65 

0.5 

0.04 

0.91 

$998,912 

$24,496,952 

G2 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

1 

2 

1 

246 

342 

Avg  Flow  Days 

127.18 

126.62 

126.55 

127.29 

Multi-skilled  Cost 

Total  Cost 

6 

P  Value  (a  =  .05) 

0.11 

0.004 

0.003 

0.18 

$1,011,245 

$24,717,231 

G3 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

1 

3 

1 

249 

345 

Avg  Flow  Days 

126.19 

125.58 

126.39 

126.23 

Multi-skilled  Cost 

Total  Cost 

9 

P  Value  (a  =  .05) 

0.35 

0.01 

0.68 

0.41 

$1,023,577 

$24,937,511 

G4 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

2 

3 

1 

252 

348 

Avg  Flow  Days 

124.99 

124.77 

124.74 

125.05 

Multi-skilled  Cost 

Total  Cost 

12 

P  Value  (a  =  .05) 

0.13 

0.02 

0.03 

0.18 

$1,035,909 

$25,157,790 

G5 

Total  Technicians 

Added 

Additional  Techs/Shift 

1 

3 

3 

1 

252 

351 

Avg  Flow  Days 

123.65 

124.07 

123.96 

124.31 

Multi-skilled  Cost 

Total  Cost 

15 

P  Value  (a  =  .05) 

0.007 

0.1 

0.05 

0.3 

$1,035,909 

$25,365,737 

G6 

Total  Technicians 

Added 

Additional  Techs/Shift 

2 

3 

3 

1 

255 

354 

Avg  Flow  Days 

123.21 

122.97 

123.25 

123.19 

Multi-skilled  Cost 

Total  Cost 

18 

P  Value  (a  =  .05) 

0.2 

0.05 

0.24 

0.18 

$1,048,241 

$25,586,017 

G7 

Total  Technicians 

Added 

Additional  Techs/Shift 

2 

4 

3 

1 

258 

357 

Avg  Flow  Days 

122.54 

122.07 

121.07 

122.49 

Multi-skilled  Cost 

Total  Cost 

21 

P  Value  (a  =  .05) 

0.006 

0.02 

0.006 

0.22 

$1,060,574 

$25,806,296 

Baseline  Flow  Days  w/  No  Personnel  Reduction 

123.33 

Flow  Day  Target  w/  OT 

125.61 

The  125.61  flow  day  target  with  overtime  is  reached  in  Level  G3,  requiring  249 
multi-skilled  personnel  and  345  personnel  at  an  annual  cost  of  $24,717,937.  The  baseline 
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future  state  (with  no  personnel  reduction)  model  flow  day  average  of  123.33  is 
considered  met  in  Level  G5  with  252  multi- skilled  personnel  and  35 1  total  personnel,  42 
personnel  less  than  the  baseline  model.  The  experiment  concludes  at  Level  G7  with  an 
average  flow  day  output  of  121.07  days,  4.5  days  below  the  flow  day  target  with 
overtime.  Following  the  Personnel  Reduction  Analysis,  Targeted  Multi-skilling 
Analysis,  and  Multi-skilled  Personnel  Add  Analysis  the  results  of  the  Future  State 
Analyses  are  compiled  for  comparison  and  summary. 

Future  State  -  Results  Summary 

Comparison  of  the  current  state  results  compiles  results  data  from  Tables  14,  16, 
and  17.  The  results  also  include  additional  model  run  outputs  for  efficiency  reductions 
associated  with  a  multi- skilled  labor  force.  Each  run  decreases  efficiency  from  98 
percent  down  to  90  percent  for  the  Targeted  Multi-skilling  Personnel  Add  best  flow  day 
scenarios  from  Table  16  and  17. 

Table  18  provides  a  list  of  the  best  results  from  each  experimental  design  as  well 
as  results  from  across  the  analyses  with  similar  flow  day  outputs.  The  delta  column 
indicates  the  difference  from  the  flow  day  targetwithovertime  based  on  the  Banded  Flow 
Day  Suggestions  provided  by  F-22  Planners.  The  annual  throughput  column  indicates 
the  aircraft  throughput  per  year,  calculated  by  taking  the  total  throughput  of  aircraft  for 
the  10-year  run  length  and  dividing  by  10.  The  cost  column  indicates  the  total  annual 
cost  as  calculated  in  the  previous  experiments. 
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Table  18:  Future  State  -  Results  Summary 


Future  State  Experiments-  Results  Summary 

Experiment  Name  (Level) 

Flow  Days 

Delta  w/ 
Target  Flow 
Days 

Annual 

Throughput 

Multi-skilled 

Personnel 

Total 

Personnel 

Annual  Cost 

E9  Multi-skill  Personnel  Add  Best  (G7) 

121.07 

-4.5 

22.04 

257 

357 

$25,806,296 

E6  Multi-skill  Personnel  Add  Best  (F4) 

121.89 

-3.7 

22.08 

348 

348 

$25,552,422 

E9  Multi-skill  Personnel  Add  (G6) 

122.97 

-2.6 

21.86 

255 

354 

$25,586,017 

E6  Multi-skill  Personnel  Add  (F3) 

123.16 

-2.5 

21.82 

345 

345 

$25,332,143 

Baseline  Model  (No  Reduction) 

123.33 

-2.3 

21.81 

0 

396 

$27,449,037 

E9  Multi-skill  Personnel  Add  (G5) 

123.65 

-2.0 

21.74 

252 

351 

$25,365,737 

E9  Multi-skill  Personnel  Add  Best  (G7)  98%  Efficient 

124.47 

-1.1 

21.61 

257 

357 

$25,806,296 

E6  Multi-skill  Personnel  Add  Best  (F4)  98%  Efficient 

124.75 

-0.9 

21.57 

348 

348 

$25,552,422 

E9  Multi-skill  Personnel  Add  Best  (G7)  95%  Efficient 

128.31 

2.7 

21.02 

257 

357 

$25,806,296 

E6  Multi-skill  Personnel  Add  Best  (F4)  95%  Efficient 

128.46 

2.9 

20.92 

348 

348 

$25,552,422 

Personnel  Reduction  5%  (Dl) 

129.08 

3.5 

20.86 

0 

378 

$26,201,354 

Personnel  Reduction  10%  (D2) 

132.92 

7.3 

20.28 

0 

357 

$24,745,723 

E6  Multi-skill  Personnel  Add  Best  (F4)  90%  Efficient 

135.5 

9.9 

19.88 

348 

348 

$25,552,422 

E9  Multi-skill  Personnel  Add  Best  (G7)  90%  Efficient 

135.88 

10.3 

19.82 

257 

357 

$25,806,296 

Average  Flow  Day  Target  w/Overtime 

125.61 

The  summarized  results  show  that  multi-skilled  workforce  scenarios  with  no 
efficiency  loss  and  between  37  and  5 1  fewer  personnel  produce  results  superior  to  the 
baseline  model  with  no  personnel  reductions.  Additionally,  the  second  best  E6 
experiment  produces  statistically  equivalent  flow  day  outputs  to  the  baseline  with  a 
reduction  in  workforce  of  51  personnel  and  a  cost  avoidance/savings  of  $2,1 16,984 
annually  or  $10,584,470  over  5  years.  The  third  best  E9  scenario  provides  very  similar 
flow  day  and  cost  avoidance/savings  results  as  the  second  best  E6  scenario. 

Even  with  a  2  percent  increase  in  all  daily  maintenance-processing  times,  the  best 
multi-skilled  scenarios  beat  flow  day  targets  by  approximately  1  day.  A  5  percent 
increase  in  all  processing  tunes  as  reflected  in  the  95  percent  efficiency  scenarios  miss 
the  flow  day  target  by  2.7  and  2.9  days.  However,  both  scenarios  provide  a  cost 
avoida  nee/ savings  of  at  least  $1,642,741  annually  or  $8,213,705  over  5  years.  The  cost 
difference  provides  space  to  add  overtune  or  personnel  to  compensate  for  the  increased 
processing  times. 
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The  experimentation  and  analysis  of  the  current  and  future  state  models  provide 
valuable  insights  into  the  benefits  associated  with  multi-skilling  personnel  within  the  F- 
22  Heavy  Maintenance  and  Modification  Program  The  researcher  makes  several 
conclusions  and  recommendations  based  the  benefits  of  multi-skilling  identified  in  this 
research. 
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V.  Conclusions  and  Recommendations 


Answers  to  Research  Questions 

The  research  ends  where  it  began,  seeking  to  answer  the  research  questions 
developed  in  collaboration  with  F-22  Heavy  Maintenance  Modification  Program 
leadership.  The  following  paragraphs  provide  answers  and  recommendations  based  on 
the  research  questions  posed  and  findings  within  this  research. 

Research  Question  1:  Which  maintenance  specialties  within  F-22  Depot  Operations 
should  be  considered  for  multi-skilling  into  the  low  observable  (LO)  specialty? 

The  results  and  analysis  of  this  research  identify  several  pairing  opportunities  for 
multi-skilling.  Aircraft  throughput,  employee  utilization,  and  cost  savings  support  the 
recommendation  for  multi- skilled  pairing  of  personnel  in  Low  Observable  specialty  with 
Sheet  Metal,  Aircraft  Mechanics  with  Aircraft  Electricians,  and  Fuels  technicians  with 
Avionics  technicians.  Additionally,  aircraft  throughput  and  flow  day  averages  show  that 
system  performance  decreases  only  slightly  when  pairing  Aircraft  Mechanics  with 
Avionics  technicians  and  Electricians  with  Fuels  technicians. 

Furthermore,  the  research  provides  the  insight  that  the  smaller  capacity  specialties 
benefit  the  system  more  through  pairings  with  similar  capacity  specialties  and  in  the  same 
manner  pairing  the  larger  specialties  (LO  and  Sheet  Metal)  show  the  most  benefit  to 
system  performance.  This  finding  is  supported  by  the  Current  State  Personnel  Add 
experiments. 

The  Current  State  Personnel  Add  experiments  show  that  overtime  hour  additions 
have  a  greater  return  on  investment  for  the  smaller  specialties  of  Aircraft  Mechanics, 
Aircraft  Fuels  Systems  Mechanics,  and  Avionics  technicians  than  for  Painters  (LO). 
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Much  larger  increases  in  overtime  hours  are  required  before  the  benefits  outweigh  adding 
to  the  smaller  specialties.  These  insights  support  the  finding  that  pairing  smaller 
specialties  with  LO  hinder  system  performance  because  the  smaller  specialties  do  not 
provide  enough  additional  labor  hours  to  affect  overall  system  performance  and 
negatively  influence  the  requirements  of  the  smaller  specialties.  This  finding  also 
indicates  that  future  improvement  efforts  should  focus  on  the  smaller  specialties  first 
because  small  improvements  in  these  areas  result  in  greater  impacts  to  system 
performance. 

Research  Question  2:  How  does  multi-skilling  increase  aircraft  throughput  and 
employee  utilization  in  F-22  Depot  Operations? 

In  both  current  and  future  operational  states  of  the  system,  targeted  multi-skilling 
policies  with  no  productivity  losses  increase  aircraft  throughput  to  levels  exceeding  the 
current  overtime  policies  with  significant  cost  savings/avoidance.  Furthermore,  the 
research  shows  that  the  best  performing  multi-skilling  policies  are  cost  lavorable  for  a 
given  level  of  aircraft  throughput  up  to  the  95  percent  efficiency  level.  Meaning  that 
multi-skilling  is  favorable  up  to  the  point  that  all  maintenance  requirements  processing 
times  increase  by  5  percent  due  to  productivity  losses  associated  with  a  more  generalized 
workforce.  At  that  point,  overtime  and  multi-skilling  become  cost  equivalent  with  further 
processing  time  increases  causing  overtime  to  be  more  desirable. 

Additionally,  utilization  of  employees  and  available  labor  hours  improves 
significantly  with  multi-skilling,  as  is  evident  by  the  experimental  results  showing 
significant  improvements  in  aircraft  flow  day  and  throughput  measures  with  a  multi- 
skilled  workforce.  In  both  the  current  and  future  state  experiments,  a  multi- skilled 
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workforce  provided  more  annual  throughput  and  direct  labor  hour  usage  than  a  workforce 
of  equal  or  greater  size  with  no  multi-skilling. 

Research  Question  3:  To  what  extent  does  multi-skilling  offset  overtime  use  to  meet 
aircraft  flow  day  requirements  and  throughput  targets? 

As  discussed  in  answering  Question  2,  a  multi- skilled  workforce  exceeds  the  flow 
day  and  throughput  targets  with  cost  performance  superior  to  overtime.  Additionally,  this 
research  supports  the  assertion  included  in  the  Multi- trade  Demonstration  Project  BCA, 
highlighting  that  multi-skilling  provides  the  opportunity  to  “increase  the  amount  of 
Organizational  and  Intermediate  level  work  on  the  aircraft  while  it  is  already  opened  for 
maintenance,  thus  reducing  future  number  of  iterations  of  LO  removal  and  restoration 
process  at  the  base  level”  (Ogden  Air  Logistics  Center,  2012,  p.  19).  The  potential 
benefits  to  the  operational  customer  should  be  included  in  the  conversation  regarding 
overtime  reductions,  cost  savings,  and  increased  throughput. 

Finally,  the  results  support  the  hypothesis  of  productivity  increases  and  cost 
savings  from  multi-skilling  proposed  in  the  BCA  completed  by  Ogden  ALC  in  2012. 
However,  the  results  indicate  that  multi-skilling  all  five  of  the  non-LO  FWS  maintenance 
specialties  into  LO,  as  prescribed  by  the  BCA,  is  not  a  desirable  policy  in  terms  of  flow 
day  reductions,  annual  aircraft  throughput,  and  cost. 

Limitations 

The  limitations  of  this  study  are  that  cost  calculations  of  multi-skilling  (multi¬ 
trade)  did  not  include  considerations  for  clearances  or  additional  training.  Additionally, 
the  aggregate  level  of  data  used  and  overuse  of  multi-skilled  personnel  associated  with 
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the  ARENA  14®  seize  limitation  hinders  optimization  of  multi-skilled  personnel. 

Finally,  tune  constraints  and  lengthy  model  run  time  limited  the  analysis  to  one  factor  at  a 
time  adjustments,  which  do  not  give  a  full  picture  of  the  interactions  and  interdependency 
of  input  variables  and  them  impact  on  outputs. 

Future  Research 

This  research  concludes  by  identifying  opportunities  for  further  research.  AFMC 
should  undertake  future  research  in  four  areas.  The  five  areas  include: 

1)  Simulation  of  future  weapon  system  depot  processes  during  the  design  phase 
to  assess  personnel  requirements  and  potential  multi-skilling  levels  and 
benefits.  The  researcher  would  recommend  looking  for  software  with  more 
fidelity  and  capabilities  in  terms  of  resource  related  scheduling  and  seizing 
(possibly  an  agent-based  or  object  oriented  simulation  software). 

2)  Methods  for  implementing  a  flexible  (multi-skilled)  workforce  to  decrease  the 
impact  proficiency  losses  associated  with  more  generalization. 

3)  Many  previous  studies  and  reports  identify  data  accuracy  issues  associated 
with  maintenance  task  times.  Further  research  should  look  at  the  level  and 
amount  of  data  needed  for  useful  decision-making  and  research  efforts  within 
the  ALC’s  and  broader  Air  Force  aircraft  maintenance  arena.  If  everyone 
knows  the  task  level  data  is  inaccurate,  there  maybe  targets  of  opportunity  to 
decrease  the  amount  of  data  inputs  required  by  technicians  and  still  capture 
the  level  of  information  needed. 
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4)  The  Clemson  MORS  tool  should  be  further  developed  to  incorporate  over  and 
above  task  hours  into  the  actual  hour  data  pulled  in.  Other  depot  maintenance 
activities  outside  of  the  F-22  program  would  benefit  from  this  tool  in  a  version 
modified  to  fit  their  processes. 

5)  Further  research  should  focus  on  ways  to  reduce  non- value  added  time  of 
maintenance  technicians  to  increase  availability  of  direct  labor  hours. 
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Appendix  A-Model  Description 


Full  Model  Description 

The  following  sections  detail  the  full  description  and  development  of  the  model 
built  in  ARENA  14®  for  this  research.  The  simulation  model  developed  in  ARENA  14® 
is  based  on  the  conceptual  model  from  Figure  3. 

The  F-22  (entity  of  interest)  flows  through  eight  main  sub  models  including 
Characteristics  and  a  separate  sub  model  for  each  of  the  Gates  1  through  7  as  depicted  in 
Figure  20:  Top  Level  Model  For  the  current  state  baseline  model  the  aircraft  arrives 
every  19  days  based  on  a  target  annual  throughput  of  aircraft  with  225  workdays  per  year. 
Once  the  aircraft  is  processed  in  all  eight  main  sub  models  it  exits  the  system  and  is 
disposed  of  in  the  Depart  process.  The  variable  views  showing  zeros  track  the  variables 
indicated  by  the  title  above  each  one  as  the  model  runs. 


Top  Level  Model 


PlaneNumber 


0 


Process  Time 


0.0 


Current  WIP 

o- 


Total  A/C 

0 


Flow  Days  Average  Flow  Days 


Figure  20:  Top  Level  Model 
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Characteristics  Sub  Model 


After  entity  creation  the  aircraft  flows  into  the  Characteristics  sub  model  as  seen 
in  F  igure  2 1 . 


The  aircraft  flows  through  the  Awaitinglnput  module  and  is  held  until  the  number 
of  work- in-process  (WIP)  aircraft  in  the  system  is  less  than  the  maximum  WIP  constraint 
using  Equation  11.  The  WIP  Constraint  variable  is  used  as  the  maximum  WIP  constraint 
and  the  value  is  initially  set  at  six  to  reflect  the  current  constraint  of  the  real  system. 

(Gatel.WIP  +  Gatel.WlP  +  Gate3.WIP  +  GateA.WIP  + 

(9) 

GateS.WIP  +  Gate6.WIP  +  Gate7.WIP)  <  WIP_Constraint 

The  aircraft  then  flows  into  the  Assign  ID  process  and  is  assigned  a  unique 
identification  (ID)  number,  a  unique  ID  variable,  and  a  plane  picture  for  animation 
purposes.  Next,  the  aircraft  is  counted  as  a  depot  input  and  the  decision  is  made  on  the 
type  ofplane  arriving  based  onrandom  chance  with  a  certain  probability  of  the  four 
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different  types  of  arrivals  being  selected.  The  decide  module  uses  ARENA’S  standard 
random  number  generation  stream  and  is  the  only  stochastic  input  in  the  model.  The 
random  selection  of  the  four  types  of  aircraft  induces  man-hour  requirement  variability 
into  the  system  and  is  a  critical  factor  to  accurately  reflect  the  variability  of  real  system 
inputs.  The  four  plane  types  are  used  in  Gates  1  through  7  sub  models  to  assign  man¬ 
hour  requirements  for  maintenance  processing  times.  The  decision  on  the  number  of 
plane  types  and  selection  of  hourly  requirements  is  discussed  later  in  the  Inputs  portion  of 
this  chapter. 

Once  the  decision  is  made  on  the  type  of  plane  arriving  it  is  assigned  attributes  for 
the  type  of  plane  (PlaneNum),  arrival  day  (ArrivalDay),  and  arrival  time  (ArrivalTime). 
Figure  22  shows  the  parameters  in  the  Assign_Planel  module  and  is  representative  of  the 
other  three  plane  types  with  the  exception  of  the  PlanN  um  value. 


Figure  22:  Assign_Plane1 

Then  the  type  of  plane  is  recorded,  a  general  maintenance  dock  is  seized  and  the 
aircraft  exits  the  Characteristics  sub  model  and  is  defined  as  work- in-process  (WIP). 


Ill 


Figure  23:  Record_Plane1 


Figure  24:  Mx_Dock_Seize1 


Gate  1  Sub  Model 

Transformations  from  inputs  into  outputs  through  the  completion  of  maintenance 
hour  requirements  begin  in  Gate  1.  The  previous  sub  model  is  used  to  determine  the 
plane  type  or  PlaneNum  attribute  based  on  the  lour  possible  plane  types  and  then 
constrains  the  system  to  a  defined  WIP  value.  Gate  1  uses  the  attributes  defined  in  the 
Characteristics  sub  model  for  decisions  on  processing  times  per  specialty  as  the  aircraft 
moves  from  one  day  to  the  next.  The  Gate  1  sub  model  (shown  in  Figure  25)  is  broken 
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down  into  six  main  processes,  two  main  sub  models  with  several  record  functions  used  to 


tally  statistics  throughout  the  gate. 


The  next  few  paragraphs  discuss  the  first  four  processes  and  the  ProcessGl  sub 
model  in  Gate  1  and  are  depicted  in  Figure  26. 
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The  first  process  upon  entering  Gate  1,  Gatel  Start,  assigns  four  attributes  to  the 
aircraft  including:  ContinueFlag  =  1,  GatelStartDay  =  CalDayOfYear(TNOW), 
Gatel_Begin=  TNOW,  and  ProcessTiine.  All  attributes  and  variables  are  described  in 
detail  in  the  process  in  which  they  are  used  in  order  to  aid  in  understanding.  The  only 
part  of  this  module  that  changes  in  Gates  2-7  is  the  numerical  character  depicting  the 
current  gate.  For  example,  in  Gate  2  the  Gatel  StartDay  attribute  is  now 
Gate2_StartDay  and  subsequent  gates  reflect  the  same  change  in  numerical  characters  to 
correspond  to  the  gate  the  aircraft  currently  resides.  One  thing  to  highlight  is  that  the 
ContinueFlag  attribute  must  be  assigned  at  the  beginning  of  each  gate  in  order  for  the 
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VBA  code  and  loop  function  (discussed  later  in  the  VBA  &  ContinueAV?  descriptions) 
to  work  correctly  within  each  gate.  Next,  the  aircraft  is  separated  into  six  dilferent 
entities  using  five  separate  blocks  and  then  flows  into  separate  paths  representing  the  six 
FWS  specialties  of  interest. 

In  queuing  theory  terms,  each  path  represents  a  different  type  of  server  required  to 
complete  a  specific  type  of  maintenance.  The  system  has  six  types  of  servers  (FWS 
maintenance  specialties)  that  can  do  maintenance  and  using  a  different  path  design  allows 
resource  pooling  based  on  the  type  of  server.  When  multi-skilling  is  introduced  the 
server  is  able  to  complete  maintenance  hours  associated  with  multiple  resource  pools 
versus  just  one.  The  paths  are  replicas  of  each  other  with  minor  equation  differences. 
Only  the  AV  (avionics)  path  is  discussed  in  detail  to  avoid  duplication  and  the  differences 
of  the  paths  are  subsequently  detailed.  For  consistency  purposes,  the  separated  aircraft  is 
referred  to  as  entities  and  again  described  as  an  aircraft  when  they  are  joined  back 
together. 

The  separated  entity  flows  into  an  assign  module,  GatelAV  depicted  in  Figure 
26  in  which  a  global  variable  (AV  Available)  and  attribute  (Counter)  are  assigned. 


Assign 

ITTbT 

Name: 

Gate1_AV 

- 

Assignments: 


:  !  -  .  -  :  ::  ■  i.ETSUMlAVIQNIi 


Attribute,  Counter,  Counter  + 1 
< End  of  list> 


OK  [  Cancel  ]  Help 


Figure  27:  Gatel  AV 


Add... 

I  E*--  -I 

|  Delete  | 
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The  AVAvailable  variable  assigns  the  current  value  based  on  the  number  of  avionics 
technicians  available  in  the  set  of  resources  called  AVIONICS  using  Equation  12. 

A  V  _  Available  =  Number  Available  -  NumberBusy  (1 0) 

The  AVIONICS  set  includes  the  avionics  personnel  (AV  resource)  and  a  separate 
multi-skill  resource  for  each  of  the  five  other  specialties  paired  with  AV  (AP_AV, 
AS_AV,  etc).  The  benefits  of  using  a  set  are  discussed  in  detail  later  in  the  maintenance 
process  description  of  the  methodology.  The  Counter  attribute  is  given  a  value  +1  and 
adds  one  to  the  count  every  time  the  entity  flows  through  the  assign  module. 

VBA  Block 

Following  the  assign  module  the  aircraft  advances  to  a  visual  basic  application 
(VBA)  block.  The  block  runs  custom  VBA  code  that  first  finds  the  path  for  an  excel 
document  (Model  Input  Data.xls)  containing  man-hour  requirements  by  specialty  and 
flow  day  for  four  different  aircraft.  The  document  needs  to  be  openprior  to  running  the 
model  to  minimize  run  tune  associated  with  continually  opening  the  document.  The  run 
time  issues  are  discussed  further  in  the  verification  section. 

Next,  the  VBA  code  looks  to  the  tab  for  one  of  the  four  aircraft  based  on  the 
PlanNum  attribute.  Then  it  starts  at  the  first  day  of  requirements  for  the  given  specialty 
path  the  aircraft  is  in  and  assigns  the  man-hour  requirement  value  to  the  ProcessTime 
attribute.  Concurrently  the  code  references  a  different  row  and  cell  for  the  following 
flow  day  that  identifies  the  start  of  the  next  gate.  It  changes  the  ContinueF  lag  attribute 
value  to  zero  when  the  next  flow  day  is  the  start  of  the  next  gate.  Once  the  ProcessTime 
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and  ContinueFlag  values  are  assigned,  the  aircraft  proceeds  to  a  sub  model  called 
AVProcessGl. 

AVProcessGl  Sub  Model 

The  AV  ProcessGl  sub  model  performs  two  main  functions  1)  it  decides  on  the 
number  of  maintenance  personnel  to  seize  in  order  to  fulfill  daily  man-hour  requirements 
for  the  aircraft  and  2)  it  seizes  the  aircraft  and  personnel  resources  for  the  required 
numbers  of  hour  to  meet  the  days  maintenance  requirements.  The  path  through  the  four 
processes  is  depicted  Figure  28. 


Figure  28:  AV  ProcessGl  (Gate  1  Sub  Model) 


The  first  step  is  deciding  on  whether  the  aircraft  has  maintenance  requirements  for 
the  current  flow  day.  The  TimeCheckAVIGl  process  decides  if  maintenance 
requirements  exist  on  the  aircraft  for  the  day.  If  the  ProcessTime  attribute  is  greater  than 
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zero  then  the  aircraft  flows  on  to  AVI  Number  process,  otherwise  it  bypasses  the  other 
processes  and  exits  the  sub  model.  The  bypass  is  important  because  the  aircraft  could  be 
held  in  the  queue  to  be  processed  in  subsequent  blocks  even  though  it  had  no 
requirements  for  the  day.  Once  the  decision  is  made,  aircraft  requiring  maintenance  flow 
into  the  AVI  Number  sub  model  shown  in  Figure  29  for  manpower  allocation  decisions. 


Figure  29:  AVI  Number  (AV  ProcessGI  Sub  Model) 


The  basic  function  of  AV 1  Number  sub  model  is  to  assign  the  number  of 
personnel  to  be  seized  to  meet  man-hour  requirements  for  the  day  based  on  the  number  of 
personnel  available.  If  none  are  available  it  holds  the  aircraft  in  the  sub  model  until 
personnel  become  available.  Before  any  decisions  are  made  the  aircraft  is  delayed  for 
one  minute  in  the  AV  Delay  SeizeDecisionl  sub  model  so  previously  held  aircraft  can 
exit  the  sub  model  when  personnel  become  available.  The  delay  is  important  because  it 
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allows  aircraft  held  in  the  sub  model  to  clear  once  personnel  become  available  prior  to 
the  next  aircraft  initiating  resource  allocation  decisions,  preserving  the  first- in- first- out 
priority  processing  of  the  model.  The  aircraft  then  flows  through  several  dec  is  ion  points 
in  order  to  avoid  allocating  more  personnel  than  SMEs  defined  as  the  maximum  number 
of  personnel  that  would  be  assigned. 

The  first  step  is  to  calculate  the  maximum  number  of  personnel  the  aircraft  needs 
lor  maintenance  processing.  The  AVMinSeizel  module  assigns  the  maximum  number  of 
personnel  needed  to  complete  the  man-hour  requirements  in  1  shift  (AVMinSeize 
attribute)  using  the  function  shown  in  Equation  13. 

MX(l,ANINT(ProcessTime/  (  OPMD  *  NumberofShifts  ) ))  (11 ) 

The  function  uses  the  larger  value  of  1  or  the  rounded  integer  solution  of 
ProcessTime  divided  by  the  product  of  OPMD  and  NumberofShifts.  In  review,  the 
ProcessTime  attribute  is  the  maintenance  hours  required  for  the  current  flow  day,  the 
OPMD  variable  is  the  output  per  man-day  of  one  employee  in  the  model,  and 
NumberofShifts  is  the  constraint  lor  the  minimum  number  of  shifts  the  man-hours  lor  the 
day  can  be  completed.  Therefore,  the  AVMinSeize  value  is  the  maximum  number  of 
personnel  required  to  complete  the  flow  day’s  man-hour  requirements  without  falling 
below  the  minimum  time  the  aircraft  should  be  in  work  each  flow  day  as  identified  by 
SMEs.  For  example,  if  the  aircraft  requires  16  AV  hours  with  an  eight  OPMD  and  one 
lor  NumberolShifts,  the  AVMinSeize  value  would  be  two.  Following  the  AVMinSeizel 
process  the  aircraft  proceeds  to  the  Decide AV_Seizel  process. 
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The  DecideAVSeizel  process  directs  the  aircraft  down  one  of  two  paths  based 
on  Equation  14.  If  the  number  ofpersonnel  available  is  less  thanthe  seize  constraint 
identified  by  SMEs,  the  equation  is  true  and  the  aircraft  flows  forward  to  the 
AV  Available  DecideMinSeizel  process.  Otherwise,  it  is  directed  down  the  second  path 
to  the  AV  Constraint  DecideMinSeizel  process. 

AVAvailable  <=  AVSeizeConstraint  (12) 

In  the  AV_Available_DecideMinSeizel  process,  if  the  AV_MinSeize  attribute  is 
greater  than  or  equal  to  the  AV  Available  variable,  Equation  15  is  true  and  the  aircraft 
flows  to  the  AV_NumberAvailableGl  process  where  it  assigns  the  AV  SeizeNum 
attribute  with  the  AV_Available  value. 

AVMinSeize  >=  AVAvailable  (13) 

If  false,  the  aircraft  proceeds  to  the  AVAvailableMinConstraintGl  process 
where  the  AV  SeizeNum  attribute  is  assigned  with  the  value  of  the  AV_MinSeize 
attribute  previously  assigned  in  the  AV_MinSeizel  process.  This  decision  ensures  that 
the  AV  SeizeNum  value  does  not  exceed  the  number  ofpersonnel  currently  available  in 
the  AV  set  (AV  Available).  The  aircraft  then  flows  to  the  AV  SeizeReloopl  decision 
process  where,  if  the  AV  SeizeNum  attribute  is  less  than  a  variable  called 
AVMinSeizeConstraint,  Equation  16  is  true  and  it  is  directed  to  a  hold  process 
(AV  Holdl)  until  AV  personnel  become  available.  If  lalse,  the  aircraft  exits  the  AVI 
Number  sub  model  because  enough  personnel  are  available  to  start  maintenance. 

AVMinSeize  <  AVMinSeizeConstraint  (14) 
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When  the  aircraft  is  directed  down  the  second  or  false  path  from 
Decide AV_Seizel  process,  it  proceeds  to  the  AVConstraintDecideMinSeizel  process 
for  a  similar  decision  to  the  first  path  with  differences  only  in  the  second  variables  used 
from  Equation  14  to  Equation  15.  In  this  decision,  if  Equation  17  is  true  the  aircraft 
flows  to  the  AV_ConstraintGl  and  the  AV  SeizeNum  attribute  is  assigned  with  a  value 
equal  to  the  AVSeizeConstraint  variable.  The  AVSeizeConstraint  variable  is  identified 
by  SMEs  as  the  maximum  number  ofpersonnel  from  the  AV  specialty  that  are  assigned 
at  any  point  in  time  to  work  one  aircraft  in  the  real  system.  If  Equation  17  is  false,  the 
aircraft  is  directed  to  the  AV_MinConstraintGl  process  where  the  AV  SeizeNum 
attribute  is  assigned  with  a  value  equal  to  the  AVMinSeize  attribute.  The  purpose  of  this 
decision  is  to  ensure  the  AV  SeizeNum  is  constrained  to  a  maximum  number  based  on 
SME  inputs  while  still  assigning  the  highest  value  possible.  The  aircraft  exits  the  AVI 
Number  sub  model  once  the  AV  SeizeNum  is  assigned  and  proceeds  to  the  AV_D1 
process  to  complete  maintenance  requirements. 

AVMinSeize  >=  AVMinSeizeConstraint  (15) 


AVDl  Process 

The  AVDl  process  is  the  heart  of  this  simulation  research.  The  aircraft  flows 
into  the  process  and  is  delayed  while  maintenance  man-hour  requirements  are  met  for  the 
day.  This  process  and  its  sister  processes  in  the  other  specialty  paths  are  the  only  process 
modules  that  convert  inputs  into  outputs  within  the  system  The  input  is  considered  an 
aircraft  with  maintenance  requirements  and  an  output  is  considered  an  aircraft  with  all 
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maintenance  man-hour  requirements  met.  All  other  processes  up  to  this  point  are  used  to 
assign  attributes  and  variables  to  facilitate  this  process. 

Process  1  B  |1  S3  | 

Name:  Type: 

AV_D  1  ▼  [  Standard  ▼  ] 

Logic 

Action:  Priority: 

[  Seize  Delay  Release  Medium(2)  ▼ 

Resources: 

Add... 

1  Ed»"_J 

Delete 

Delay  Type:  Units:  Allocation: 

[  Constant  ▼  |  [  Hours  ▼  [  [  Value  Added  ▼  ] 

Value: 

ProcessTime/  ( EfficiencyF. 

</  Report  Statistics 

OK  |  [  Cancel  |  Help 

Figure  30:  AV_D1  Process 

The  process  uses  a  standard  type  of  module  with  a  Seize  Delay  Release  action  and 
assigns  resources  by  Set.  The  set  name  is  AVIONICS_AV  and  includes  the  AV  resource 
and  the  five  additional  multi-skill  resources.  The  quantity  used  is  the  AV  SeizeNum 
attribute  assigned  in  the  AVI  Number  sub  model.  The  selection  rule  is  Preferred  Order 
and  the  order  is  set  for  all  runs  subsequent  to  the  baseline  run  with  the  priority  of  first 
seizing  resources  that  have  lower  utilization  rates  in  the  baseline  outputs.  The  resource 
types  in  the  set  and  selection  rule  order  are  depicted  in  F igure  3 1 .  The  order  uses  the 
baseline  model  and  focuses  pairings  to  seize  lower  utilized  resources  first  using  the 
resource  utilization  rates.  Identifying  the  quantity  of  resources  (personnel)  seized  is  a 
critical  step  before  moving  on  to  the  calculation  of  the  processing  (delay)  time  value. 


Set  AVIONICS  AV,  AV  SeizeNum,  Preferred  Order, 


122 


Figure  31:  AVIONICS  AV  Resource  Set 

The  amount  of  time  the  aircraft  is  delayed  (processing  time)  is  based  the  man¬ 
hours  required  for  AV  on  the  current  flow  day  (Process  Time),  the  number  of  personnel 
assigned  to  complete  maintenance  (AV  SeizeNum),  and  an  efficiency  factor  for  the  pool 
of  resources  used  (EflficiencyFactorAV).  Equation  7,  from  Chapter  3,  depicts  the 
function  used  to  decide  the  value  of  the  process  delay. 

ProcessTime/  (  Efficiency!7  actor  AV  *  AV  SeizeNum)  (7) 

The  efficiency  factor  is  used  to  adjust  the  efficiency  of  the  resource  pool  with 
multi-skilling.  The  initial  value  of  the  resource  pool  is  one  and  can  be  adjusted  down 
(.99,  .98,  etc.)  to  depict  a  loss  in  skill  or  efficiency.  Levien  (2010)  multiplies  processing 
times  by  a  cross  training  factor  in  his  multi-skill  research  on  the  KC- 135  PDM  process  in 
order  to  emulate  longer  task  durations  associated  with  multi-skilled  employees 
completing  maintenance  tasks.  The  efficiency  factor  in  this  research  mathematically 
produces  similar  results  by  increasing  task  times  as  efficiency  (task  proficiency) 
decreases.  However,  multi-skilling  an  employee  stipulates  that  the  employee  is  fully 
qualified  at  the  journeyman  level  on  both  skill  sets  (Federal  Service  Impasses  Panel, 
1997)  and  therefore  equally  proficient.  The  baseline  model  and  experimental  runs  are 
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initially  run  with  an  efficiency  factor  of  one.  The  analysis  section  of  this  chapter  explains 
the  sensitivity  analysis  methods  used  to  gauge  the  impacts  on  model  outputs  of 
decreasing  skill  within  the  labor  pool.  Once  the  maintenance  man-hours  are  met  for  the 
current  flow  day  the  aircraft  is  released  and  proceeds  to  the  AVLaborHoursGl  module. 


AVLaborHoursGl 

The  AV  LaborHoursGl  module  assigns  the  two  global  variables,  AVUse  and 
AV  LaborHours  as  depicted  in  Figure  32. 


Assign 

Name: 

AV_LaborHoursG1 

Assignments: 

Variable,  AVJJse,  AV_Use  +  AV  SeizeNurr 
Variable,AV  LaborHours,  AV  LaborHours 

El 


V  II  S3  | 


Add... 

E* 

Delete 


OK  Cancel  Help 


Figure  32:  AV  LaborHoursGl 


The  global  (system)  trait  of  both  variables  is  significant  because  the  function  acts 
like  a  tally  ofthe  number  of  AV  technicians  seized  and  the  AV  labor  hours  that  are  used 
in  the  system  Equation  18  defines  the  value  assigned  to  the  AV_Use  variable.  The 
AV_Use  variable  reflects  the  current  global  (system)  value  and  the  aircraft’s  current  AV 
SeizeNum  attribute  value  is  added  to  the  global  (system)  value  of  AV_Use  every  time  an 
aircraft  passes  through  the  module.  The  AV  Use  variable  then  reflects  the  total  number 
of  AV  personnel  seized  up  to  that  point  within  the  system 

AV  _Use  +  A  VSeizeNum  (16) 
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The  AVLaborHours  function  is  similar  and  tallies  the  total  AV  labor  hours  used 
by  adding  the  ProcessTime  attribute  value  of  the  aircraft  passing  through  the  module. 

The  ProcessTime  attribute  value  is  the  current  flow  day  AV  man-hours  that  were  just 
completed  in  the  AV  Dl  process.  Equation  19  shows  the  value  assigned  to  the 
AV  LaborHours  variable.  Remember  that  each  gate  has  a  duplicate  module  with  the 
same  functions  with  each  one  concurrently  adding  to  the  global  values  of  the  variables. 
Additionally,  each  specialty  path  is  the  same  and  the  variables  are  continuously 
calculated  for  each  of  the  six  specialties  (AP_Use,  AP_LaborHours,  AS_Use,  AS_Labor 
Hours,  etc.)  in  each  of  the  seven  gates.  The  aircraft  exits  the  AV  ProcessGl  submodel 
once  the  labor  hours  and  number  of  personnel  seized  is  recorded. 

A  V  _  LaborHours  +  Pr  ocessTime  (17) 

The  next  few  paragraphs  discuss  the  last  sub  model  and  process  modules  in  Gate 
1  as  depicted  Figure  33. 


125 


In  summary,  the  second  half  of  the  gate  restrains  or  aligns  the  six  identical  aircraft 
in  order  to  stay  within  a  certain  number  of  flow  days  of  each  other.  It  then  re-  loops  the 
aircraft  back  to  the  beginning  of  the  gate  when  the  aircraft  requires  more  flow  days 
within  the  gate  and  combines  the  aircraft  back  into  one  entity  before  allowing  it  to  exit 
the  current  gate.  The  mechanism  aligning  flow  days  by  aircraft  is  the  first  process  in  the 
second  half  of  the  gate  and  occurs  within  the  sub  model  GatelMatch. 

GatelMatch 

The  first  process  in  the  second  half  of  the  gate  is  the  GatelMatch  sub  model 
depicted  in  Figure  34.  This  process  is  important  because  it  restricts  the  six  duplicates  of 
one  aircraft  to  within  a  certain  number  of  flow  days  of  each  other  based  on  SME 
consultations.  LoopConstraint  is  the  variable  created  and  used  to  reflect  the  SME  defined 
constraint  for  how  far  ahead  maintenance  specialties  can  work  from  other  specialties 
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working  the  same  aircraft.  Figure  34  shows  all  six  identical  specialty  paths  but  the  AV 
path  equation  is  the  only  one  described  in  detail  for  brevity  purposes. 


The  aircraft  enters  the  sub  model  and  enters  the  Decide AVI  Loop  where  the 
decision  is  made  on  whether  the  aircraft  needs  to  be  realigned  to  the  same  flow  day  of  its 
duplicates  or  whether  it  will  bypass  the  subsequent  matching  functions.  The 
DecideAVI  Loop  module’s  defined  parameters  are  depicted  in  Figure  35. 
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Decide  j  V  H~SS~ 

Name:  Type: 

DecideAV1_Loop  ▼  [  2-way  by  Condition  ^ ) 

If: 


Value: 


Counler/LoopConslrainl  ==  AINTfCounfer/LoopConstraint) 

OK  |  [  Cancel  |  |  Help 

Figure  35:  Decide  AV1_Loop 

The  parameters  route  the  aircraft  for  matching  when  the  Counter  attribute  divided 
by  the  LoopContraint  variable  is  equal  to  the  truncated  value  of  the  same  function.  In 
order  for  Equation  20  to  be  true,  the  Counter  divided  by  the  LoopConstraint  has  to  equal 
an  integer  value  because  the  truncated  value  is  always  an  integer.  For  example,  if  the 
Counter  value  is  6  and  the  LoopConstraint  is  2,  Equation  21  would  be  true  because  the 
quotient  is  an  integer  of  3.  If  the  Counter  value  is  5  with  the  same  LoopConstraint  of  2, 
the  equation  would  be  false  because  the  quotient  of  2.5  does  not  equal  the  truncated 
quotient  of  2.  If  the  equation  is  false,  the  aircraft  bypasses  the  matching  function  and 
exits  the  sub  model.  The  aircraft  that  are  routed  for  matching  proceed  to 
Gatel_Separatel6  module. 


Counter  /  LoopConstra  int  ==  A I  NT  ( Counter  /  LoopConstra  int)  (18) 


The  Gatel_Separatel6  module  creates  a  duplicate  of  the  aircraft  and  routes  it  to 
the  Gatel  LoopBatch  module.  The  original  flows  down  a  separate  path  to  the 
GatelAVMatch  module  where  it  is  held.  The  Gate  1  Loop  Batch  module  holds  the 
duplicate  aircraft  until  the  other  five  aircraft  in  the  other  specialty  paths  containing  the 
same  IDNumber  attribute  (assigned  in  Characteristics  sub  model)  arrive.  Remember  that 
all  six  specialty  paths  are  identical  so  the  other  five  model  specialty  paths  (AT,  AG,  AP, 
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AR,  and  AS)  create  a  duplicate  in  the  same  way  as  AV.  The  batch  function  holding  the 
entity  until  all  six  of  the  same  IDN umber  arrives  realigns  the  duplicates  to  the  same  flow 
day.  The  separate  but  identical  entities  are  batched  together  and  allowed  to  proceed  as 
one  to  the  GatelSeparatel  1  module. 

In  the  GatelSeparatel  1  module  the  entity  is  duplicated  with  the  one  of  the 
duplicates  flowing  to  the  Gatel  AVMatch  module  and  the  other  duplicate  proceeds  to 
another  Separate  module.  The  duplicate  entity  that  flows  to  the  Gatel  AVMatch 
module  is  matched  to  the  original  entity  that  is  being  held  for  matching  with  another 
entity  with  the  same  IDN umber.  Figure  36  identifies  the  parameter  values  for  matching 
in  the  Gatel  AVMatch  module. 

Match  1  ?  H  S3  | 

Name:  Number  to  Match: 

Gate1_AVMatch  ▼  1 2  ▼  | 

Type:  Attribute  Name: 

|  Based  on  Attribute  |  IDNumber  ▼ 

OK  J  [  Cancel  |  Help 

Figure  36:  Gate1_AVMatch 

Once  an  aircraft  with  the  same  IDNumber  arrives,  both  aircraft  are  released  and 
the  original  aircraft  exits  the  sub  model  while  the  duplicate  is  routed  to  the 
Gate  1M ate hLoopDispose  module  for  disposal.  An  important  detail  to  highlight  is  that 
the  original  aircraft  retains  the  same  attributes  that  it  entered  the  GatelMatch  sub  model 
with  but  the  duplicate  only  retains  the  unique  IDN  umber  because  of  the  batching.  This  is 
important  because  the  attribute  values  of  the  original  must  be  retained  for  future  use.  The 
subsequent  separate  module  and  duplicate  aircraft  simultaneously  repeat  the  process  and 
then  flow  to  another  Separate  module  until  all  six  have  been  reduplicated,  matched,  and 
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released.  Functionally,  this  process  realigns  all  six  specialties’  entities  to  the  same  flow 
day  and  does  so  every  so  many  flow  days  as  defined  by  the  LoopConstraint.  Once  the 
flow  days  are  realigned  the  entity  flows  out  of  the  sub  model  to  another  decide  module  in 
Gate  1. 


ContinueAV?  Module 

The  next  decide  module  Continue  A  V?  decides  whether  or  not  the  entity  has  more 
flow  day  requirements  in  the  current  gate.  The  parameters  displayed  in  Figure  37  re- loop 
the  entities  back  to  GatelAV  module  if  the  ContinueFlag  attribute  is  one.  The  aircraft 
then  recompletes  the  Gate  1  AV  path. 


Figure  37:  Continue  AV? 


Recall  that  the  VBA  block  assigns  a  value  of  zero  to  the  ContinueFlag  if  the  next 
flow  day  is  the  start  of  the  next  gate.  When  the  value  is  zero  and  the  entity  enters  the 
Continue  AV?  module,  the  condition  is  lalse  and  the  entity  is  directed  to  the  exit  path  of 
the  Gate  1  sub  model 
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Gate  1  Exit  Path 


The  exit  path  of  each  gate  routes  through  five  modules  before  exiting  Gate  1 .  The 
modules  include  a  batching  module,  three  record  modules,  and  an  assign  module  as 
sho  wn  in  F  igure  3  8 . 


Figure  38:  Gate  1  AV  Re-Loop  and  Exit  Path 

The  first  module  on  the  exit  path  is  the  Gatel  AV  Hours  record  module  with 
parameters  depicted  in  Figure  39.  It  tallies  the  number  of  hours  each  entity  spends  in  the 
Gate  1  AV  path  by  subtracting  the  Gatel  Begin  attribute  from  the  current  time  (TNOW). 


Record 

f  11  S3 

Name: 

Type: 

Gale1_AV_Hours 

▼  Expression 

_ H 

Value: 


TNOW  -  Gale1_Begin 

Tally  Name: 

0  Record  into  Sel 

Gate1_AV_Hours 

▼ 

OK  Cancel  Help 


Figure  39:  Gate1_AV_Hours 
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The  value  recorded  and  reported  in  output  values  is  not  used  for  this  research 
because  the  Gatel  Match  sub  model  aligns  the  time  spent  in  each  path  and  in  most  cases 
the  value  will  be  the  same  for  all  six  paths.  However,  if  the  LoopConstraint  value  in 
GatelMatch  is  given  a  high  enough  value,  it  will  no  longer  be  a  constraint  and  the  entities 
on  the  six  paths  with  the  same  IDNumber  will  no  longer  be  realigned  to  the  same  flow 
day. 

Next,  the  entity  flows  to  the  GatelBatch  module  and  is  held  until  the  other  five 
entities  with  the  same  IDNumber  arrive.  Then  entities  are  combined  back  into  one 
aircraft  and  it  is  released  to  the  final  two  record  modules  and  an  assign  module. 


Figure  40:  Gatel  Batch 

The  final  two  record  modules  are  used  to  tally  the  total  days  and  the  total  hours 
the  aircraft  is  in  the  gate.  GatelDays  module  records  the  number  of  days  using  the 
parameters  depicted  in  Figure  41  dividing  the  difference  of  the  current  simulation  time 
minus  the  gate  start  time  by  the  FIrsPerDay  variable. 
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mrisT 
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Gate1_Day$ 

Expression 

▼ 

Value: 

|  TNOW  •  Gate1_Begin  )  /HrsPerD 
Tally  Name: 

Gate1_Day$ 


J  Record  into  Set 


OK 


Cancel 


Help 


Figure  41:  Gatel  Days 


Gatel  Hours  tallies  the  number  of  hours  the  aircraft  spent  in  the  gate  using  the 
parameters  depicted  in  Figure  42,  subtracting  the  gate  start  time  by  the  current  simulation 
time. 


Figure  42:  Gate1_Hours 


The  aircraft  is  then  assigned  a  plane  picture  in  AssignPlaneGl  module  and  exits 
Gatel  and  proceeds  to  Gate  2.  There  are  a  couple  differences  between  Gate  1  and  Gates 
2  through  7  that  need  to  be  clearly  defined  in  order  to  fully  understand  the  functions  and 
outputs  of  the  model. 


Gate  Sub  Model  Differences 

In  addition  to  the  equation  differences  previously  described,  Gates  1-7  sub  models 
contain  small  differences  in  the  exit  path  in  order  to  collect  additional  output  statistics 
and  also  to  reflect  constraints  and  additional  actions  identified  by  SMEs.  The  following 
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paragraphs  begin  by  identifying  the  modules  used  to  collect  additional  statistics  and  then 
discuss  unique  modules  and  processes  added  in  certain  gates  in  order  to  accurately  reflect 
the  real  system 

Gate  2-7  Exit  Path  Differences 

Gates  2  through  7  have  an  additional  record  module  after  the  Gate#_Hours 
module  named  Gate#_Cumulative  days.  The  #  reflects  the  gate  number  the  module 
resides.  This  module,  with  parameters  shown  in  Figure  43,  records  the  total  flow  days 
the  aircraft  is  in  the  system  by  taking  the  difference  of  current  simulation  time  (TNOW) 
minus  the  ArrivalTime  attribute  and  dividing  it  by  the  HoursPerDay  variable.  This 
statistic  shows  the  time  it  takes  lor  an  aircraft,  in  total  cumulative  days,  to  meet  the 
maintenance  requirements  through  each  gate.  For  example,  the  Gate2_CumulativeDays 
value  shows  how  long  an  aircraft  takes  to  get  from  the  start  of  Gate  1  to  the  finish  of  Gate 
2. 


Record 

[ 

9  l|  S3  | 

Name: 

Type: 

Gate2_Days 

- 

|  Expression 

d 

Value: 

l  TNOW  •  Gate2_Begin )  )  HrsPerD 

d  Record  into  Set 

Tally  Name: 

Gate2_Days 

* 

OK 

Cancel 

Help 

Figure  43:  Gate2_CumulativeDays 


Gate  4  Differences 

Gates  4  has  two  additional  modules  in  the  exit  path.  The  first  additional  module, 
Gate4MinHold,  is  used  to  hold  the  aircraft  in  Gate  4  until  it  meets  the  12  day  constraint 
identified  by  SMEs  as  the  minimum  number  of  days  it  takes  to  complete  the  gate.  The 
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constraint  is  given  a  variable  name  Gate4MinDays  with  a  value  of  12.  A  variable  is  used 
for  ease  of  adjustment  to  the  value.  Specialties  outside  of  the  six  considered  in  this 
research  complete  a  majority  of  the  work  and  SMEs  identified  that  the  maintenance  hour 
requirements  do  not  reflect  the  time  it  takes  the  aircraft  to  complete  the  gate.  Figure  44 
shows  the  parameters  and  equation  used  in  Gate4MinFIold  to  hold  and  release  the 
aircraft. 


Hold 

|  3  ||  S3  | 

Name: 

Type: 

Gale4MinHold 

▼  Scan  for  Condition  ▼ 

Condition: 

{  T  N  0 W  -  G  ale4  B  egin  )  /  H  rsPerD  ay  >  =  G  ale4M  inD  ays  1  If 

Queue  Type: 

Queue 

Queue  Name: 

G  ate4M  inH  old.  Q  ueue 

’r 

OK  |  |  Cancel  |  |  Help 


Figure  44:  Gate4MinHold 

The  second  additional  module,  Mx_Dock_ReleaseG4,  releases  the  general 
maintenance  dock  seized  in  the  Characteristics  sub  model  using  the  parameters  depicted 
in  Figure  45. 
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Process 


19  S3 


Figure  45:  Mx_Dock_ReleaseG4 

Gate  5  &  6  Differences 

Gate  5  and  6  contain  the  seize  and  release  modules  for  LO  docks,  reflecting  the 
aircraft  being  moved  into  low  observable  maintenance  (LO)  at  the  beginning  of  Gate  5 
and  exiting  LO  maintenance  at  the  end  of  Gate  6.  The  LO_Dock_SeizeG5  module  is  the 
first  process  in  Gate  5  and  seizes  an  LO  dock.  The  LO  Dock  Re lease G6  module  is  the 
last  process  in  Gate  6  and  uses  the  same  function  as  the  seize  function  in  Gate  5  except  it 
releases  the  LO  dock  in  the  Action  preference.  The  last  of  the  differences  within  the 
gates  is  seen  in  Gate  7. 

Gate  7  Differences 

Gate  7  is  shown  in  Appendix  A  and  contains  several  minor  differences  from  the 
Gate  1  description.  The  differences  start  with  the  Mx_Dock_SeizeGate7  module,  the  first 
process  within  the  gate.  It  seizes  a  general  maintenance  dock  in  the  same  way  the 
Mx  Dock  Seizel  module  does  at  the  end  of  the  Characteristics  sub  model.  A  similar 
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difference  occurs  at  the  end  of  Gate  7  with  the  Mx_Dock_ReleaseGate7  module,  where 
the  aircraft  releases  the  general  maintenance  dock. 

The  completed  aircraft  then  flows  through  the  FlowDays  record  module  to  tally 
the  final  flow  day  value.  This  value  is  reported  in  the  output  statistics  and  reflects  the 
average  flow  days  needed  to  complete  an  aircraft.  Next,  the  DepotExitCount  record 
module  counts  the  aircraft.  The  depot  exit  count  reflects  aircraft  throughput  for  the 
model  run.  The  Flow  Days  and  Depot  Exit  Count  outputs  are  the  most  significant 
outputs  for  this  research.  Following  the  calculation  of  these  values  the  aircraft  proceeds 
to  the  final  difference  between  Gate  7  and  the  other  gates,  the  PlaneTypeStats  sub  model. 

The  PlaneTypeStats  sub  model  shown  in  Figure  46  is  the  finalprocess  prior  to  the 
F-22  departing  from  the  depot.  The  function  of  this  sub  model  is  to  tally  the  number  of 
flow  days  of  each  of  the  four  types  of  aircraft  that  flow  through  the  depot. 


First,  the  aircraft  is  directed  down  the  path  corresponding  to  the  type  of  aircraft  it 
is  using  the  DecidePlaneTypeStats  module.  The  decision  is  based  on  the  PlaneNum 
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attribute  and  directs  the  aircraft  down  one  of  four  paths  using  the  parameters  shown  in 


Figure  47. 


Figure  47:  DecidePlaneTypeStats 

Then  the  aircraft  flows  through  one  of  the  Plane#Flow_Days  record  modules 
based  on  the  path  it  is  directed  down.  The  flow  days  are  calculated  in  the  same  way  as 
previous  flow  day  record  module  with  the  difference  being  that  the  outputs  are  based  on 
the  type  of  plane  and  not  all  plane  types.  This  is  important  because  this  sub  model 
creates  flow  day  outputs  based  on  the  aircraft  type  and  will  provide  valuable  insights  into 
the  differences  in  average  flow  days  based  on  the  man-hour  requirements  of  the  aircraft. 
Once  the  values  are  recorded  the  aircraft  exits  Gate  7  and  departs  the  system  by  flowing 
into  the  Depart  disposal  module. 

The  model  of  an  aircraft  flowing  through  the  F-22  Heavy  Maintenance 
Modification  Program  is  discussed  in  the  previous  sections  with  a  focus  on  the  process  of 
transforming  an  input  aircraft  into  a  finished  output  aircraft.  Over  the  next  two  sections  I 
discuss  the  ten  separate  models  that  continually  calculate  global  variable  values  and  to 
record  the  final  values  of  the  statistics  and  global  variables  of  interest  for  this  study. 
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These  separate  models  fell  into  one  of  two  categories,  either  set  availability  calculation 
models  or  final  value  record  models. 

Set  Availability  Calculation  Models 

In  a  discrete- event  simulation  the  model  is  event  driven  and  the  simulation  tune 
proceeds  from  one  event  to  the  next.  This  characteristic  presents  a  problem  in  this 
research  due  to  the  way  the  way  the  model  is  designed  to  hold  during  the  AVINumber 
sub  model.  The  hold  function  does  not  create  an  event  to  show  when  the  hold  condition 
is  met  and  the  global  variable  values  will  only  change  from  one  event  to  the  next  or  when 
an  entity  is  assigned  a  global  variable  value  (AV_Available,  AT_Available,  etc.).  This 
creates  the  need  for  a  model  that  has  an  event  to  recalculate  the  global  variable  value  at  a 
certain  interval. 

Figure  48  depicts  the  six  models  that  are  built  to  recalculate  the  global  (system) 
variable  values  for  each  of  the  six  FWS  specialties  sets  (COATERAP, 

SHEET  METAL  AS,  AVIONICS  AV,  etc.).  This  grouping  of  models  is  hidden  under 
Gate  1  to  keep  the  Top  Level  Model  as  visually  simple  as  possible.  The  naming 
convention  corresponds  to  the  variable  that  is  being  calculated  and  the  corresponding  set 
to  which  it  is  used.  The  next  paragraph  details  the  specifications  of  the  AV  model  but  the 
others  are  identical  with  the  exception  of  the  variable  being  calculated. 
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Figure  48:  Set  Availability  Calculation  Models 


The  AV  model  creates  one  entity  at  the  start  of  the  simulation  run  and  the  entity 
loops  through  the  model  for  the  entirety  of  the  run.  Upon  creation  the  entity  flows 
through  the  Loop  AV  module  and  is  assigned  the  global  variable  AV  Available  in  the 
same  way  it  is  assigned  in  the  Gatel  AV  module  at  the  start  of  Gate  1  ’s  AV  path. 
Assigning  this  variable  causes  the  variable  to  be  recalculated  each  time  the  entity  flows 
through  the  module. 

After  the  current  value  for  AV_Available  is  assigned  /  recalculated  the  entity 
proceeds  to  the  AV  LoopDelay  module  and  is  delayed  for  a  number  of  hours  based  on 
the  variable  Availability  Loop  Delay.  The  same  variable  is  used  for  all  six  models  so  the 
delay  is  consistent  within  each.  The  value  input  for  the  AvailabiltiyLoopDelay  is  one 
hour  so  that  every  hour  the  entity  is  released  and  the  AV_Available  variable  value  is 
recalculated.  Functionally  the  model  creates  an  event  every  hour  to  recalculate  the  global 
variables  used  for  set  availability  values  and  allows  for  the  hold  functions  used  in  the 
resource  allocation  sub  models  (AVI  Number,  AV2  Number,  etc.)  to  release  the  entities 
when  personnel  become  available.  The  final  discussion  points  in  the  Model  Description 
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section  of  this  chapter  are  the  models  created  to  calculate  the  final  values  of  variables  of 
interest  and  statistics  derived  from  those  values. 

Final  Value  Record  Models 

The  built  in  statistics  in  ARENA  14®  are  used  to  calculate  the  final  record 
module  statistics  associated  with  the  main  system  model  but  these  statistics  do  not 
encapsulate  the  entirety  of  outputs  desired.  Additional  outputs  desired  include  the  total 
labor  hours  used  by  specialty,  the  total  number  of  times  a  specialty  was  used  (number  of 
seizes),  the  number  of  times  multi-skilled  personnel  are  used,  and  the  percentage  of  the 
time  multi-skilled  personnel  are  used  versus  the  original  labor  pooL  The  following 
paragraphs  discuss  the  model  built  to  create  and  calculate  these  outputs. 

The  model  creates  one  entity  at  the  final  simulation  time  (TFIN),  flowing  the 
entity  through  four  record  modules  per  FWS  specialty  totaling  24  modules  as  shown  in 
Figure  49.  The  four  AV  modules  are  described  in  detail  for  brevity  and  consistency 
purposes.  The  four  modules  include  AV  LaborHoursTotal,  AV  Multi-skill  UseTotal, 
AV  UseTotal,  and  AVMulti-skillUsePercent.  The  other  specialties’  modules  contain 
identical  equations  except  for  the  unique  specialty  variables  used. 
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Figure  49:  Final  Value  Record  Model 


The  AVLaborHoursTotal  module  simply  tallies  the  final  value  of  the 
AV  LaborHours  function.  The  corresponding  output  value  represents  the  total  number 
of  on  aircraft  (direct)  labor  hours  used  for  the  AV  specialty  during  the  simulation  run 
The  AVUseTotal  module  records  the  final  value  of  the  AVUse  variable  and  reflects  the 
total  number  of  AV  resource  seized  during  the  simulation  run  to  complete  maintenance 
actions.  The  final  two  outputs  generated  involve  calculating  multi- skill  use. 

The  AVMulti-skillUseTotal  calculates  the  number  of  instances  when  a  multi- 
skilled  resource  (technician)  is  seized  to  complete  a  maintenance  requirement  instead  of 
the  primary  specialty.  The  output  value  reflects  the  AV  Use  variable  minus  the  number 
of  times  the  primary  AV  resource  was  seized  (ResSeizes(AV)).  Refer  to  Figure  49  for 
the  parameters  used  within  the  module. 
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Figure  50:  AVMulti-skillJJseTotal 


The  AV  Multi-skill  UsePercent  module  uses  the  AV  Multi-skill  UseTotal  value 
function  and  divides  it  by  the  AV_Use  variable  to  get  the  AV_Multi-skill_UsePercent 
output  value.  The  percentage  that  multi-skill  resources  are  utilized  outside  of  their 
primary  specialty  is  important  because  of  the  25%  threshold  dictated  by  OPM  as  the 
minimum  amount  of  tune  multi- trade  personnel  have  to  be  used  in  the  secondary  skill 
(Office  of  Personnel  Management,  n.d.). 

ARENA  clears  its  internal  statistics  after  the  warm-up  period  of  the  simulation  but 
the  variable  values  created  for  this  research  are  not  cleared  in  the  same  way.  The  model 
shown  in  F  igure  5 1  is  created  to  clear  the  statistics  at  the  end  of  the  warm-up  period.  It 
creates  one  entity  at  the  warm-up  period  prescribed  in  validation  and  flows  through 
modules  that  set  the  global  variable  values  back  to  zero.  After  resetting  the  values  back 
to  zero  the  entity  is  eliminated  from  the  model  via  a  dispose  module. 


a 

m 

Jj  Stat  Clear 

Figure  51:  Statistic  Reset  Model 
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This  concludes  the  full  description  of  the  model  developed  for  this  research. 
Additional  figures  and  tables  follow  to  support  the  methodology  in  Chapter  3  and  the 
model  developed. 
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Appendix  B-  Validation  Data 


Flow  Day  Validation 

Table  19:  Current  State  Validation 


Current  State  Validation:  Expected  vs  Model  Flow  Days 

Aircraft  Category 

Man-Hour 

Requirement 

Expected 
Non-OT  Days 

Model 

Output 

Delta 

%  Delta 

Plane  1 

20471.60 

144.60 

141.92 

-2.68 

-1.86% 

Plane  2 

20127.10 

142.17 

141.69 

-0.48 

-0.34% 

Plane  3 

19900.60 

140.57 

138.80 

-1.77 

-1.26% 

Plane  4 

15663.10 

110.64 

121.69 

11.05 

9.99% 

Avg  Aircraft 

18617.06 

131.50 

133.99 

2.49 

1.89% 

Average  Aircraft  Flow  Days 

95%  Confidence  Interval 

Lower 

95%  Cl 

133.01 

Upper 

95%  Cl 

134.97 

Table  20:  Future  State  Validation 


Future  State  Validation:  Expected  vs  Model  Flow  Days 

Aircraft  Category 

Man-Hour 

Requirement 

Expected 
Non-OT  Days 

Model 

Output 

Delta 

%  Delta 

Plane  1 

20471.60 

144.60 

127.77 

-16.83 

-11.64% 

Plane  2 

20127.10 

142.17 

127.42 

-14.75 

-10.38% 

Plane  3 

19900.60 

140.57 

123.81 

-16.76 

-11.92% 

Plane  4 

15663.10 

110.64 

109.67 

-0.97 

-0.88% 

Avg  Aircraft 

19384.41 

136.92 

123.33 

-13.59 

-9.93% 

Average  Aircraft  Flow  Days 

95%  Confidence  Interval 

Lower 

95%  Cl 

122.75 

Upper 

95%  Cl 

123.91 

145 


Appendix  C-  Analysis  Results 


Current  State-  Manpower  Add 


~  Oneway  Analysis  of  Flow days  By  Treatment 
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Figure  53:  Manpower  Current  State  -  Experiment  A1  JMP®  ANOVA 


l[S  Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
21  Confidence  Quantile 
q*  Alpha 

3.01047  0.05 

>  LSD  Threshold  Matrix 

>  Connecting  Letters  Report 
2)  Ordered  Differences  Report 


Level 

AP  Treatment 
Baseline  Treatment 
AR  Treatment 
ATTreatment 
AV  Treatment 
AS  Treatment 
AP  Treatment 
Baseline  Treatment 
AP  Treatment 
AP  Treatment 
Baseline  Treatment 
Baseline  Treatment 
AR  Treatment 
AP  Treatment 
AR  Treatment 
AR  Treatment 
Baseline  Treatment 
ATTreatment 
AVTreatment 
AP  Treatment 
ATTreatment 


-  Level 

AG  Treatment 
AG  Treatment 
AG  Treatment 
AG  Treatment 
AG  Treatment 
AG  Treatment 
AS  Treatment 
AS  Treatment 
AVTreatment 
ATTreatment 
AVTreatment 
ATTreatment 
AS  Treatment 
AR  Treatment 
AVTreatment 
ATTreatment 
AR  Treatment 
AS  Treatment 
AS  Treatment 
Baseline  Treatment 
AVTreatment 


Difference 

4.486384 

4.175325 

3.715065 

3.063808 

2.967092 

2.634345 

1.852039 

1.540980 

1.519293 

1.422576 

1.208233 

1.111517 

1.080720 

0.771319 

0.747973 

0.651257 

0.460260 

0.429463 

0.332747 

0.311059 

0.096717 


Std  Err  Dif 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 

0.6935743 


Lower  CL 
2  39840 
2.08734 
1.62708 
0.97582 
0.87911 
0.54636 
-0.23595 
-0.54700 
-0.56869 
-0.66541 
-0.87975 
-0.97647 
-1.00727 
-1.31667 
-1.34001 
-1.43673 
-1.62772 
-1.65852 
-1.75524 
-1.77693 
-1.99127 


Upper  CL 

6.574369 

6.263310 

5.803050 

5.151793 

5.055077 

4.722330 

3.940024 

3.628965 

3.607278 

3.510561 

3.296219 

3.199502 

3.168705 

2.859304 

2.835958 

2.739242 

2.548245 

2.517449 

2.420732 

2.399044 

2.184702 


p-Value 


Figure  54:  Manpower  Current  State  -  Experiment  A1  JMP®  Tukey-Kramer  HSD 
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I  t  Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
a  Confidence  Quantile 
q*  Alpha 

3.01047  0.05 

l>  LSD  Threshold  Matrix 
>  Connecting  Letters  Report 
A  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Upper  CL 

p-Value 

AR  Treatment 

AG  Treatment 

2.752354 

0.7738193 

0.42279 

5.081914 

0.0101* 

/ 

AP  Treatment 

AG  Treatment 

2.526164 

0.7738193 

0.19660 

4.855724 

0.0246* 

'  :  :  :  / 

Baseline  Treatment 

AG  Treatment 

2.404190 

0.7738193 

0.07463 

4.733750 

0.0384* 

1  i  / 

AR  Treatment 

AT  Treatment 

2.251207 

0.7738193 

-0.07835 

4.580767 

0.0652 

1  :  :  :/ 

AP  Treatment 

AT  Treatment 

2.025017 

0.7738193 

-0.30454 

4.354577 

0.1322 

_ :  : 

Baseline  Treatment 

AT  Treatment 

1.903043 

0.7738193 

-0.42652 

4.232603 

0.1859 

AVTreatment 

AG  Treatment 

1.879455 

0.7738193 

-0.45011 

4.209015 

0.1979 

AR  Treatment 

AS  Treatment 

1.858262 

0.7738193 

-0.47130 

4.187822 

0.2092 

AP  Treatment 

AS  Treatment 

1.632072 

0.7738193 

-0.69749 

3.961632 

0.3556 

Baseline  Treatment 

AS  Treatment 

1.510098 

0.7738193 

-0.81946 

3.839659 

0.4520 

AVTreatment 

AT  Treatment 

1.378308 

0.7738193 

-0.95125 

3.707869 

0.5638 

:  :  j  : 

AVTreatment 

AS  Treatment 

0.985364 

0.7738193 

-1.34420 

3.314924 

0.8624 

AS  Treatment 

AG  Treatment 

0.894091 

0.7738193 

-1.43547 

3.223652 

0.9088 

AR  Treatment 

AVTreatment 

0.872899 

0.7738193 

-1.45666 

3.202459 

0.9179 

/  : 

AP  Treatment 

AVTreatment 

0.646709 

0.7738193 

-1.68285 

2.976269 

0.9806 

•  / 

Baseline  Treatment 

AVTreatment 

0.524735 

0.7738193 

-1.80483 

2.854295 

0.9935 

:  : 

:  | 

AT  Treatment 

AG  Treatment 

0.501147 

0.7738193 

-1.82841 

2.830707 

0.9950 

AS  Treatment 

AT  Treatment 

0.392945 

0.7738193 

-1.93662 

2.722505 

0.9987 

:  : 
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Baseline  Treatment 

0.348164 

0.7738193 

-1.98140 

2.677724 

0.9993 
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AP  Treatment 
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Figure  56:  Manpower  Current  State  -  Experiment  A2  JMP®  Tukey-Kramer  HSD 
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Figure  57:  Manpower  Current  State  -  Experiment  A3  JMP®  ANOVA 


-  Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
^  Confidence  Quantile 
q*  Alpha 

3.01047  0.05 

t>  LSD  Threshold  Matrix 
t>  Connecting  Letters  Report 
a  Ordered  Differences  Report 


Level 

Baseline  Treatment 
AP  Treatment 
Baseline  Treatment 
AS  Treatment 
AP  Treatment 
AR  Treatment 
AV  Treatment 
AS  Treatment 
AR  Treatment 
Baseline  Treatment 
AV  Treatment 
Baseline  Treatment 
Baseline  Treatment 
AP  Treatment 
AP  Treatment 
AG  Treatment 
AP  Treatment 
Baseline  Treatment 
AS  Treatment 
AR  Treatment 
AS  Treatment 


-  Level 
AT  Treatment 
AT  Treatment 
AG  Treatment 
AT  Treatment 
AG  Treatment 
AT  Treatment 
AT  Treatment 
AG  Treatment 
AG  Treatment 
AV  Treatment 
AG  Treatment 
AR  Treatment 
AS  Treatment 
AV  Treatment 
AR  Treatment 
AT  Treatment 
AS  Treatment 
AP  Treatment 
AV  Treatment 
AV  Treatment 
AR  Treatment 


Difference 

2.157381 

1.871749 

1.746684 

1.466272 

1.461052 

1.445462 

1.274732 

1.055575 

1.034765 

0.882649 

0.864035 

0.711919 

0.691109 

0.597017 

0.426287 

0.410697 

0.405476 

0.285632 

0.191540 

0.170730 

0.020811 


Std  Err  Dif 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 

0.7548394 


Lower  CL 
-0.11504 
-0.40067 
-0.52574 
-0.80615 
-0.81137 
-0.82696 
-0.99769 
-1.21685 
-1.23766 
-1.38977 
-1.40839 
-1.56050 
-1.58131 
-1.67540 
-1.84613 
-1.86172 
-1.86695 
-1.98679 
-2.08088 
-2.10169 
-2.25161 


Upper  CL 
4.429803 
4.144170 
4.019106 
3.738694 
3.733473 
3.717883 
3.547154 
3.327997 
3.307186 
3.155071 
3.136457 
2.984341 
2.963531 
2.869439 
2.698709 
2.683119 
2.677898 
2.558054 
2.463962 
2.443151 
2.293232 


Figure  58:  Manpower  Current  State  -  Experiment  A3  JMP®  Tukey-Kramer  HSD 
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Figure  59:  Manpower  Current  State  -  Experiment  A4  JMP®  ANOVA 


r  Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
a  Confidence  Quantile 
q*  Alpha 

3.01047  0.05 

>  LSD  Threshold  Matrix 

>  Connecting  Letters  Report 
a  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Upper  CL 

p-Value 

ATTreatment 

AG  Treatment 

2.033943 

0.8009711 

-0.37736 

4.445243 

0.1569 

Baseline  Treatment 

AG  Treatment 

1.714517 

0.8009711 

-0.69678 

4.125816 

0.3377 

AS  Treatment 

AG  Treatment 

1.712465 

0.8009711 

-0.69883 

4.123765 

0.3391 

AP  Treatment 

AG  Treatment 

1.645591 

0.8009711 

-0.76571 

4.056891 

0.3879 

AR  Treatment 

AG  Treatment 

1.297162 

0.8009711 

-1.11414 

3708462 

0.6702 

ATTreatment 

AVTreatment 

1.175588 

0.8009711 

-1.23571 

3.586888 

0.7630 

AVTreatment 

AG  Treatment 

0.858356 

0.8009711 

-1.55294 

3.269655 

0.9349 

Baseline  Treatment 

AVTreatment 

0.856161 

0.8009711 

-1.55514 

3.267461 

0.9357 

AS  Treatment 

AVTreatment 

0.854109 

0.8009711 

-1.55719 

3.265409 

0.9364 

AP  Treatment 

AVTreatment 

0.787235 

0.8009711 

-1.62406 

3.198535 

0.9566 

ATTreatment 

AR  Treatment 

0.736782 

0.8009711 

-1.67452 

3.148082 

0.9685 

AR  Treatment 

AVTreatment 

0.438806 

0.8009711 

-1.97249 

2.850106 

0.9980 

Baseline  Treatment 

AR  Treatment 

0.417355 

0.8009711 

-1.99395 

2.828655 

0.9985 

AS  Treatment 

AR  Treatment 

0.415303 

0.8009711 

-1.99600 

2.826603 

0.9985 

ATTreatment 

AP  Treatment 

0.388353 

0.8009711 

-2.02295 

2.799652 

0.9990 

AP  Treatment 

AR  Treatment 

0.348429 

0.8009711 

-2.06287 

2.759729 

0.9995 

ATTreatment 

AS  Treatment 

0.321478 

0.8009711 

-2.08982 

2.732778 

0.9997 

ATTreatment 

Baseline  Treatment 

0.319427 

0.8009711 

-2.09187 

2.730727 

0.9997 

Baseline  Treatment 

AP  Treatment 

0.068926 

0.8009711 

-2.34237 

2.480226 

1.0000 

AS  Treatment 

AP  Treatment 

0.066874 

0.8009711 

-2.34443 

2.478174 

1.0000 

Baseline  Treatment 

AS  Treatment 

0.002052 

0.8009711 

-2.40925 

2.413352 

3 


Figure  60:  Manpower  Current  State  -  Experiment  A4  JMP®  Tukey-Kramer  HSD 
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Figure  61:  Manpower  Current  State  -  Experiment  A5  JMP®  ANOVA 


ji~  Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
^  Confidence  Quantile 
q*  Alpha 

3.01047  0.05 

0  LSD  Threshold  Matrix 
[  Connecting  Letters  Report 
A  Ordered  Differences  Report 


Level 

AG  Treatment 
AP  Treatment 
Baseline  Treatment 
AT  Treatment 
AG  Treatment 
AP  Treatment 
AG  Treatment 
AS  Treatment 
AP  Treatment 
AR  Treatment 
AG  Treatment 
AP  Treatment 
AG  Treatment 
AP  Treatment 
Baseline  Treatment 
Baseline  Treatment 
AT  Treatment 
AT  Treatment 
Baseline  Treatment 
AS  Treatment 
AG  Treatment 


-  Level 
AV  Treatment 
AV  Treatment 
AV  Treatment 
AV  Treatment 
AR  Treatment 
AR  Treatment 
AS  Treatment 
AV  Treatment 
AS  Treatment 
AV  Treatment 
AT  Treatment 
AT  Treatment 
Baseline  Treatment 
Baseline  Treatment 
AR  Treatment 
AS  Treatment 
AR  Treatment 
AS  Treatment 
AT  Treatment 
AR  Treatment 
AP  Treatment 


Difference 

1.815979 

1.785069 

1.202680 

1.123184 

0.980097 

0.949187 

0.913923 

0.902056 

0.883014 

0.835882 

0.692795 

0.661886 

0.613299 

0.582390 

0.366798 

0.300624 

0.287301 

0.221128 

0.079496 

0.066174 

0.030909 


Std  Err  Dif 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 

0.8312289 


Lower  CL 
-0.68641 
-0.71732 
-1.29971 
-1.37921 
-1.52229 
-1.55320 
-1.58847 
-1.60033 
-1.61938 
-1.66651 
-1.80960 
-1.84050 
-1.88909 
-1.92000 
-2.13559 
-2.20177 
-2.21509 
-2.28126 
-2.42289 
-2.43622 
-2.47148 


Upper  CL 

4.318369 

4.287460 

3.705070 

3.625574 

3.482487 

3.451577 

3.416313 

3.404446 

3.385404 

3.338272 

3.195185 

3.164276 

3.115689 

3.084780 

2.869188 

2.803014 

2.789692 

2.723518 

2.581886 

2.568564 

2.533300 


p-Value 


Figure  62:  Manpower  Current  State  -  Experiment  A5  JMP®  Tukey-Kramer  HSD 
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Figure  63:  Manpower  Current  State  -  Experiment  A6  JMP®  ANOVA 


-  Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
^  Confidence  Quantile 
q*  Alpha 

3.01047  0.05 


>  LSD  Threshold  Matrix 

>  Connecting  Letters  Report 
Ordered  Differences  Report 

Difference  StdErrDif  Lower  CL  Upper  CL  p-Value 


Level  -  Level 

Baseline  Treatment  ATTreatment 
AP  Treatment  ATTreatment 

Baseline  Treatment  AR  Treatment 
AP  Treatment  AR  Treatment 

AV  Treatment  ATTreatment 

AVTreatment  AR  Treatment 

AG  Treatment  ATTreatment 

Baseline  Treatment  AS  Treatment 
AP  Treatment  AS  Treatment 

AG  Treatment  AR  Treatment 

AV  Treatment  AS  Treatment 

AG  Treatment  AS  Treatment 

AS  Treatment  AT  Treatment 

Baseline  Treatment  AG  Treatment 
AP  Treatment  AG  Treatment 

AS  Treatment  AR  Treatment 

Baseline  Treatment  AVTreatment 
AP  Treatment  AVTreatment 

AVTreatment  AG  Treatment 

AR  Treatment  ATTreatment 

Baseline  Treatment  AP  Treatment 


1.383031 

0.6974654 

-0.71667 

3.482730 

1.370564 

0.6974654 

-0.72914 

3.470263 

1.324726 

0.6974654 

-0.77497 

3.424425 

1.312259 

0.6974654 

-0.78744 

3.411958 

1.184610 

0.6974654 

-0.91509 

3.284309 

1.126305 

0.6974654 

-0.97339 

3.226004 

1.081530 

0.6974654 

-1.01817 

3.181229 

1.060519 

0.6974654 

-1.03918 

3.160218 

1.048051 

0.6974654 

-1.05165 

3.147750 

1.023225 

0.6974654 

-1.07647 

3.122924 

0.862097 

0.6974654 

-1.23760 

2.961796 

0.759018 

0.6974654 

-1.34068 

2.858717 

0.322512 

0.6974654 

-1.77719 

2.422211 

0.301501 

0.6974654 

-1.79820 

2.401200 

0.289034 

0.6974654 

-1.81067 

2.388733 

0.264207 

0.6974654 

-1.83549 

2.363906 

0.198421 

0.6974654 

-1.90128 

2.298120 

0.185954 

0.6974654 

-1.91375 

2.285653 

0.103080 

0.6974654 

-1.99662 

2.202779 

0.058305 

0.6974654 

-2.04139 

2.158004 

0.012467 

0.6974654 

-2.08723 

2.112166 

Figure  64:  Manpower  Current  State  -  Experiment  A6  JMP®  Tukey-Kramer  HSD 
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Figure  65:  Manpower  Current  State  -  Experiment  A7  JMP®  ANOVA 


-  Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
A  Confidence  Quantile 
q*  Alpha 

3.01047  0.05 


>  LSD  Threshold  Matrix 

>  Connecting  Letters  Report 
A  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Upper  CL 

p-Value 

AV  Treatment 

AR  Treatment 

1.100461 

0.7449868 

-1.14230 

3.343222 

0.7576 

AT  Treatment 

AR  Treatment 

0.812309 

0.7449868 

-1.43045 

3.055070 

0.9296 

AP  Treatment 

AR  Treatment 

0.758180 

0.7449868 

-1.48458 

3.000941 

0.9488 

AV  Treatment 

Baseline  Treatment 

0.729131 

0.7449868 

-1.51363 

2.971891 

0.9575 

AV  Treatment 

AS  Treatment 

0.691049 

0.7449868 

-1.55171 

2.933810 

0.9672 

AG  Treatment 

AR  Treatment 

0.553210 

0.7449868 

-1.68955 

2.795970 

0.9895 

AV  Treatment 

AG  Treatment 

0.547252 

0.7449868 

-1.69551 

2.790012 

0.9901 

AT  Treatment 

Baseline  Treatment 

0.440978 

0.7449868 

-1.80178 

2.683739 

0.9969 

AS  Treatment 

AR  Treatment 

0.409412 

0.7449868 

-1.83335 

2.652173 

0.9980 

AT  Treatment 

AS  Treatment 

0.402897 

0.7449868 

-1.83986 

2.645658 

0.9981 

AP  Treatment 

Baseline  Treatment 

0.386850 

0.7449868 

-1.85591 

2.629610 

0.9985 

Baseline  Treatment  AR  Treatment 

0.371331 

0.7449868 

-1.87143 

2.614091 

0.9988 

AP  Treatment 

AS  Treatment 

0.348768 

0.7449868 

-1.89399 

2.591529 

0.9992 

AV  Treatment 

AP  Treatment 

0.342281 

0.7449868 

-1.90048 

2.585042 

0.9993 

AV  Treatment 

AT  Treatment 

0.288152 

0.7449868 

-1.95461 

2.530913 

0.9997 

AT  Treatment 

AG  Treatment 

0.259099 

0.7449868 

-1.98366 

2.501860 

0.9999 

AP  Treatment 

AG  Treatment 

0.204971 

0.7449868 

-2.03779 

2.447731 

1.0000 

AG  Treatment 

Baseline  Treatment 

0.181879 

0.7449868 

-2.06088 

2.424640 

1.0000 

AG  Treatment 

AS  Treatment 

0.143798 

0.7449868 

-2.09896 

2.386558 

1.0000 

AT  Treatment 

AP  Treatment 

0.054129 

0.7449868 

-2.18863 

2.296889 

1.0000  i 

AS  Treatment 

Baseline  Treatment 

0.038082 

0.7449868 

-2.20468 

2.280842 

1.0000 

Figure  66:  Manpower  Current  State  -  Experiment  A7  JMP®  Tukey-Kramer  HSD 
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Figure  67:  Manpower  Current  State  -  Experiment  A8  JMP®  ANOVA 


E  Comparisons  for  all  pairs  using  Tukey-Kramer  H  SD 
^  Confidence  Quantile 
q*  Alpha 

3.01047  0.05 

>  LSD  Threshold  Matrix 

>  Connecting  Letters  Report 
4)  Ordered  Differences  Report 


Level 

AT  Treatment 
AV  Treatment 
Baseline  Treatment 
AG  Treatment 
AS  Treatment 
AR  Treatment 
AT  Treatment 
AV  Treatment 
AT  Treatment 
AV  Treatment 
Baseline  Treatment 
Baseline  Treatment 
AT  Treatment 
AV  Treatment 
Baseline  Treatment 
AG  Treatment 
AG  Treatment 
AT  Treatment 
AV  Treatment 
AS  Treatment 
AT  Treatment 


-  Level 
AP  Treatment 
AP  Treatment 
AP  Treatment 
AP  Treatment 
AP  Treatment 
AP  Treatment 
AR  Treatment 
AR  Treatment 
AS  Treatment 
AS  Treatment 
AR  Treatment 
AS  Treatment 
AG  Treatment 
AG  Treatment 
AG  Treatment 
AR  Treatment 
AS  Treatment 
Baseline  Treatment 
Baseline  Treatment 
AR  Treatment 
AV  Treatment 


Difference 

6.038760 

5.917554 

5.455197 

4.507452 

3.898132 

3.586290 

2.452470 

2.331264 

2.140629 

2.019422 

1.868907 

1.557065 

1.531308 

1.410101 

0.947745 

0.921162 

0.609321 

0.583563 

0.462357 

0.311841 

0.121206 


Std  Err  Dif 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 

0.7680161 


Lower  CL 

3.72667 
3.60546 
3.14311 
2.19536 
1.58604 
1.27420 
0.14038 
0.01917 
-0.17146 
-0.29267 
-0.44318 
-0.75502 
-0.78078 
-0.90199 
-1.36435 
-1.39093 
-1.70277 
-1.72853 
-1.84973 
-2.00025 
-2  19068 


Upper  CL 

8.350850 

8.229644 

7.767287 

6.819542 

6.210222 

5.898380 

4.764560 

4.643353 

4.452718 

4.331512 

4.180997 

3.869155 

3.843398 

3.722191 

3.259835 

3.233252 

2.921411 

2.895653 

2.774447 

2.623931 

2.433296 


Figure  68:  Manpower  Current  State  -  Experiment  A8  JMP®  Tukey-Kramer  HSD 
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Figure  69:  Manpower  Current  State  -  Experiment  A9  JMP®  ANOVA 


ix  Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
A  Confidence  Quantile 
q*  Alpha 

3.01047  0.05 

>  LSD  Threshold  Matrix 
t>  Connecting  Letters  Report 
A  Ordered  Differences  Report 


Level 

AV  Treatment 
Baseline  Treatment 
AT  Treatment 
AV  Treatment 
AV  Treatment 
Baseline  Treatment 
Baseline  Treatment 
AT  Treatment 
AT  Treatment 
AV  Treatment 
Baseline  Treatment 
AT  Treatment 
AR  Treatment 
AG  Treatment 
AS  Treatment 
AR  Treatment 
AR  Treatment 
AV  Treatment 
AV  Treatment 
Baseline  Treatment 
AG  Treatment 


-  Level 
AP  Treatment 
AP  Treatment 
AP  Treatment 
AS  Treatment 
AG  Treatment 
AS  Treatment 
AG  Treatment 
AS  Treatment 
AG  Treatment 
AR  Treatment 
AR  Treatment 
AR  Treatment 
AP  Treatment 
AP  Treatment 
AP  Treatment 
AS  Treatment 
AG  Treatment 
AT  Treatment 
Baseline  Treatment 
AT  Treatment 
AS  Treatment 


Difference 

5.076897 

4.626445 

4.433300 

4.064321 

3.934330 

3.613869 

3.483879 

3.420724 

3.290733 

3.176298 

2.725846 

2.532701 

1.900599 

1.142567 

1.012576 

0.888023 

0.758032 

0.643597 

0.450451 

0.193145 

0.129990 


Std  Err  Dif 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 

0.9726369 


Lower  CL 
2.14880 
1.69835 
1.50521 
1.13623 
1.00624 
0.68577 
0.55578 
0.49263 
0.36264 
0.24820 
-0.20225 
-0.39539 
-1.02750 
-1.78553 
-1.91552 
-2.04007 
-2.17006 
-2.28450 
-2.47764 
-2.73495 
-2.79810 


Upper  CL 

8.004992 

7.554540 

7.361395 

6.992415 

6.862425 

6.541964 

6.411974 

6.348819 

6.218828 

6.104393 

5.653941 

5.460796 

4.828694 

4.070661 

3.940671 

3.816118 

3.686127 

3.571692 

3.378546 

3.121240 

3.058085 


Figure  70:  Manpower  Current  State  -  Experiment  A9  JMP®  Tukey-Kramer  HSD 
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Future  State 


Targeted  Multi -skill  Analysis 


Means  Comparisons 

A  i»  Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
J  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Upper  CL 

p-Value 

Treatment  3 

Treatment? 

9.73696 

0.4542712 

8.2312 

11.24275 

<0001* 

Baseline  Treatment  Treatment  1 

9.35254 

0.4542712 

7.8467 

10.85834 

<0001* 

Treatments 

Treatment  9 

9.29630 

0.4542712 

7.7905 

10.80210 

<0001* 

Treatment  4 

Treatments 

9.18381 

0.4542712 

7.6780 

10.68961 

<0001* 

Treatment  2 

Treatments 

8.30042 

0.4542712 

6.7946 

9.80622 

<0001* 

Treatments 

Treatment  10 

8.05731 

0.4542712 

6.5515 

9.56311 

<.0001* 

Treatments 

Treatments 

8.02719 

0.4542712 

6.5214 

9.53299 

<0001* 

Baseline  Treatment  Treatments 

7.24442 

0.4542712 

5.7386 

8.75022 

<0001* 

Treatment  2 

Treatment? 

7.20076 

0.4542712 

5.6950 

8.70656 

<.0001* 

Baseline  Treatment  Treatment  2 

6.97119 

0.4542712 

5.4654 

8.47699 

<0001* 

Treatments 

Treatment? 

6.92753 

0.4542712 

5.4217 

8.43333 

<0001* 

Treatment  2 

Treatments 

6.76011 

0.4542712 

5.2543 

8.26591 

<0001* 

Treatments 

Treatment  9 

6.48688 

0.4542712 

4.9811 

7.99268 

<0001* 

Treatments 

Treatments 

6.21555 

0.4542712 

4.7098 

7.72135 

<.0001* 

Treatment  1 

Treatments 

5.91907 

0.4542712 

4.4133 

7.42487 

<0001* 

Treatments 

Treatment  11 

5.82933 

0.4542712 

4.3235 

7.33513 

<0001* 

Treatment  2 

Treatment  10 

5.52111 

0.4542712 

4.0153 

7.02691 

<0001* 

Treatments 

Treatment  10 

5.24789 

0.4542712 

3.7421 

6.75369 

<0001* 

Treatment  11 

Treatments 

5.00728 

0.4542712 

3.5015 

6.51308 

<0001* 

Treatments 

Treatment  1 

4.91755 

0.4542712 

3.4117 

6.42335 

<0001* 

Treatment  1 

Treatment? 

4.81941 

0.4542712 

3.3136 

6.32521 

<0001* 

Treatment  4 

Baseline  Treatment 

4.74881 

0.4542712 

3.2430 

6.25461 

<0001* 

Treatments 

Treatments 

4.62106 

0.4542712 

3.1153 

6.12686 

<.0001* 

Baseline  Treatment  Treatments 

4.43500 

0.4542712 

2.9292 

5.94080 

<0001* 

Treatment  1 

Treatment9 

4.37876 

0.4542712 

2.8730 

5.88456 

<0001* 

Treatment  11 

Treatment? 

3.90762 

0.4542712 

2.4018 

5.41342 

<0001* 

Treatment  2 

Treatments 

3.67936 

0.4542712 

2.1736 

5.18516 

<0001* 

Treatments 

Treatment? 

3.52140 

0.4542712 

2.0156 

5.02720 

<0001* 

Treatment  11 

Treatment  9 

3.46697 

0.4542712 

1.9612 

4.97277 

<.0001* 

Treatments 

Treatments 

3.40613 

0.4542712 

1.9003 

4.91193 

<0001* 

Treatment  2 

Treatment  11 

3.29314 

0.4542712 

1.7873 

4.79894 

<0001* 

Treatment  1 

Treatment  10 

3.13976 

0.4542712 

1.6340 

4.64556 

<0001* 

Treatments 

Treatment9 

3.08075 

0.4542712 

1.5750 

4.58655 

<.0001* 

Treatments 

Treatment  11 

3.01991 

0.4542712 

1.5141 

4.52571 

<0001* 

Treatments 

Treatments 

2.80942 

0.4542712 

1.3036 

4.31522 

<0001* 

Treatment  10 

Treatments 

2.77931 

0.4542712 

1.2735 

4.28511 

<0001* 

Treatments 

Treatment  2 

2.53619 

0.4542712 

1.0304 

4.04199 

<0001* 

Treatment  2 

Treatment  1 

2.38135 

0.4542712 

0.8756 

3.88715 

<0001* 

Treatment  11 

Treatment  10 

2.22798 

0.4542712 

0.7222 

3.73378 

0.0001* 

Treatments 

Treatment  1 

2.10813 

0.4542712 

0.6023 

3.61392 

0.0004* 

Treatments 

Treatment  10 

1.84176 

0.4542712 

0.3360 

3.34756 

0.0043* 

Treatment  10 

Treatment? 

1.67965 

0.4542712 

0.1738 

3.18545 

0.0151* 

Treatment  9 

Treatments 

1.54031 

0.4542712 

0.0345 

3.04611 

0.0399* 

Treatment  1 

Treatments 

1.29800 

0.4542712 

-0.2078 

2.80380 

0.1664 

Treatment  10 

Treatment9 

1.23899 

0.4542712 

-0.2668 

2.74479 

0.2223 

Treatment? 

Treatments 

1.09966 

0.4542712 

-0.4061 

2.60546 

0.3982 

Treatment  1 

Treatment  11 

0.91178 

0.4542712 

-0.5940 

2.41758 

0.6880 

Treatment9 

Treatment? 

0.44065 

0.4542712 

-1.0651 

1.94645 

0.9981 

Treatment  11 

Treatments 

0.38622 

0.4542712 

-1.1196 

1.89202 

0.9994 

Treatment  2 

Treatments 

0.27323 

0.4542712 

-1.2326 

1.77903 

1.0000 

Figure  71:  Future  State  -  Targeted  Multi-skill  JMP®  Tukey-Kramer  HSD 
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Multi-skill  Add  -  15  Percent  Targeted  Experiment  6 


Oneway  Analysis  of  Flow days  By  Treatment 


Figure  72:  Future  State  -  (E6)  Add  Experiment  FI  JMP®  ANOVA 


Means  Comparisons 

a  ^Comparisons  for  each  pair  using  Student's  t 
A  Confidence  Quantile 
t  Alpha 

2.00324  0.05 

t>  LSD  Threshold  Matrix 
>  Connecting  Letters  Report 
A  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Baseline  Treatment  AR/AG 

1.678044 

0.3722968 

0.932243 

Baseline  Treatment  AP/AS 

0.991681 

0.3722968 

0.245880 

Baseline  Treatment  AT/AV 

0.907487 

0.3722968 

0.161687 

AT/AV 

AR/AG 

0.770557 

0.3722968 

0.024757 

AP/AS 

AR/AG 

0.686363 

0.3722968 

-0.059437 

AT/AV 

AP/AS 

0.084194 

0.3722968 

-0.661607 

Upper  CL  p-Value 
2.423844  <.0001* 
1.737481  0.0101* 
1.653287  0.0180* 
1.516357  0.0431* 
1.432163  0.0705 
0.829994  0.8219 


A  w  Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
<4  Confidence  Quantile 
q*  Alpha 

2.64794  0.05 

t>  LSD  Threshold  Matrix 
>  Connecting  Letters  Report 
a  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Upper  CL 

p-Value 

Baseline  Treatment  AR/AG 

1.678044 

0.3722968 

0.692223 

2.663864 

0.0002* 

Baseline  Treatment  AP/AS 

0.991681 

0.3722968 

0.005860 

1.977501 

0.0481* 

Baseline  Treatment  AT/AV 

0.907487 

0.3722968 

-0.078333 

1.893307 

0.0816 

AT/AV 

AR/AG 

0.770557 

0.3722968 

-0.215264 

1.756377 

0.1756 

AP/AS 

AR/AG 

0.686363 

0.3722968 

-0.299457 

1.672183 

0.2642 

AT/AV 

AP/AS 

0.084194 

0.3722968 

-0.901627 

1.070014 

0.9959 

Figure  73:  Future  State  -  (E6)  Add  Experiment  FI  JMP®  Student’s  t  &  Tukey-Kramer  HSD 


156 


Figure  74:  Future  State  -  (E6)  Add  Experiment  F2  JMP®  ANOVA 

Means  Comparisons 

ai~  Comparisons  for  each  pair  using  Student's  t 
a  Confidence  Quantile 
t  Alpha 
2.00324  0.05 

>  LSD  Threshold  Matrix 

>  Connecting  Letters  Report 


A  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Upper  CL 

p-Value 

Baseline  Treatment 

AP/AS 

1.307768 

0.3113245 

0.684110 

1.931426 

<0001* 

1  :  / 

Baseline  Treatment 

AR/AG 

1.226270 

0.3113245 

0.602612 

1.849928 

0.0002* 

J  mV 

Baseline  Treatment 

AT/AV 

0.727685 

0.3113245 

0.104027 

1.351343 

0.0230* 

M  1  ;  :m  : 

AT/AV 

AP/AS 

0.580083 

0.3113245 

-0.043575 

1.203741 

0.0677 

Z V.  \J\ 

AT/AV 

AR/AG 

0.498585 

0.3113245 

-0.125073 

1.122243 

0.1149 

^  : 

AR/AG 

AP/AS 

0.081498 

0.3113245 

-0.542160 

0.705156 

0.7945 

^^Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
a  Confidence  Quantile 
q*  Alpha 

2.64794  0.05 

t>  LSD  Threshold  Matrix 
[>  Connecting  Letters  Report 
a  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Upper  CL 

p-Value 

Baseline  Treatment 

AP/AS 

1.307768 

0.3113245 

0.483399 

2.132137 

0.0005* 

1  lM  :  / 

Baseline  Treatment 

AR/AG 

1.226270 

0.3113245 

0.401901 

2.050640 

0.0013* 

V  i  '-J 

Baseline  Treatment 

AT/AV 

0.727685 

0.3113245 

-0.096684 

1.552055 

0.1018 

'  ,  1  M 

AT/AV 

AP/AS 

0.580083 

0.3113245 

-0.244287 

1.404452 

0.2555 

AT/AV 

AR/AG 

0.498585 

0.3113245 

-0.325784 

1.322954 

0.3861 

IJ-  \J  \ 

AR/AG 

AP/AS 

0.081498 

0.3113245 

-0.742871 

0.905867 

0.9936 

^  y  ■■  ■■ 

Figure  75:  Future  State  - 

(E6)  Add 

Experiment  F2 

JMP®  Student’s  t  &  Tukey-Kramer 
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E  Oneway  Analysis  of  Flow_days  By  Treatment 
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Treatment  Students  t  Tukey-Kramer 


Figure  76:  Future  State  -  (E6)  Add  Experiment  F3  JMP®  ANOVA 


Means  Comparisons 

21 1  Comparisons  for  each  pair  using  Student's  t 
21  Confidence  Quantile 
t  Alpha 

2.00324  0.05 

i>  LSD  Threshold  Matrix 
t  Connecting  Letters  Report 
2i  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Upper  CL 

p-Value 

Baseline  Treatment  AR/AG 

1.076259 

0.2904826 

0.494352 

1.658166 

0.0005* 

AT/AV 

AR/AG 

0.882090 

0.2904826 

0.300183 

1.463997 

0.0036* 

Baseline  Treatment  AP/AS 

0.651240 

0.2904826 

0.069333 

1.233146 

0.0289* 

AT/AV 

AP/AS 

0.457071 

0.2904826 

-0.124836 

1.038977 

0.1212 

AP/AS 

AR/AG 

0.425019 

0.2904826 

-0.156887 

1.006926 

0.1490 

Baseline  Treatment  AT/AV 

0.194169 

0.2904826 

-0.387738 

0.776076 

0.5066 

2i  »  Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
21  Confidence  Quantile 
q"  Alpha 

2.64794  0.05 

t>  LSD  Threshold  Matrix 
>  Connecting  Letters  Report 
21  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Upper  CL 

p-Value 

Baseline  Treatment  AR/AG 

1.076259 

0.2904826 

0.307078 

1.845440 

0.0027* 

/  i ; 

AT/AV 

AR/AG 

0.882090 

0.2904826 

0.112909 

1.651271 

0.0185* 

/  1  :  y 

Baseline  Treatment  AP/AS 

0.651240 

0.2904826 

-0.117941 

1.420421 

0.1246 

1  :  /: 

AT/AV 

AP/AS 

0.457071 

0.2904826 

-0.312110 

1.226252 

0.4018 

AP/AS 

AR/AG 

0.425019 

0.2904826 

-0.344162 

1.194200 

0.4663 

in  \  J  \  \ 

Baseline  Treatment  AT/AV 

0.194169 

0.2904826 

-0.575012 

0.963350 

0.9085 

TT  :  _ 

Figure  77:  Future  State  -  (E6)  Add  Experiment  F3  JMP®  Student’s  t  &  Tukey-Kramer  HSD 
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Figure  78:  Future  State  -  (E6)  Add  Experiment  F4  JMP®  ANOVA 


Means  Comparisons 

A  ^Comparisons  for  each  pair  using  Student's  t 
a  Confidence  Quantile 
t  Alpha 

2.00324  0.05 

>  LSD  Threshold  Matrix 

>  Connecting  Letters  Report 
a  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Upper  CL 

p-Value 

Baseline  Treatment  AR/AG 

1.272537 

0.3530211 

0.565350 

1.979723 

0.0007* 

Baseline  Treatment  AP/AS 

0.744265 

0.3530211 

0.037078 

1.451451 

0.0395* 

Baseline  Treatment  AT/AV 

0.708869 

0.3530211 

0.001683 

1.416055 

0.0495* 

AT/AV 

AR/AG 

0.563667 

0.3530211 

-0.143519 

1.270854 

0.1160 

AP/AS 

AR/AG 

0.528272 

0.3530211 

-0.178914 

1.235458 

0.1402 

AT/AV 

AP/AS 

0.035396 

0.3530211 

-0.671791 

0.742582 

0.9205 

A  ▼Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
a  Confidence  Quantile 
q*  Alpha 

2.64794  0.05 

t>  LSD  Threshold  Matrix 
[>  Connecting  Letters  Report 
a  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Baseline  Treatment  AR/AG 

1.272537 

0.3530211 

0.337757 

Baseline  Treatment  AP/AS 

0.744265 

0.3530211 

-0.190515 

Baseline  Treatment  AT/AV 

0.708869 

0.3530211 

-0.225910 

AT/AV 

AR/AG 

0.563667 

0.3530211 

-0.371112 

AP/AS 

AR/AG 

0.528272 

0.3530211 

-0.406508 

AT/AV 

AP/AS 

0.035396 

0.3530211 

-0.899384 

Upper  CL  p-Value 

2.207316  0.0036* 
1.679044  0.1630 
1.643648  0.1973 
1.498447  0.3888 
1.463051  0.4464 
0.970175  0.9996 


Figure  79:  Future  State  -  (E6)  Add  Experiment  F3  JMP®  Student’s  t  &  Tukey-Kramer  HSD 
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Multi -skill  Add  -  15  Percent  Targeted  Experiment  9  (E9) 


Figure  80:  Future  State  -  (E9)  Add  Experiment  G1  JMP®  ANOVA 


Means  Comparisons 

zi  ~  Comparisons  for  each  pair  using  Student's  t 
z)  Confidence  Quantile 
t  Alpha 

1.99444  0.05 

>  LSD  Threshold  Matrix 

>  Connecting  Letters  Report 
z)  Ordered  Differences  Report 


Level 

Baseline  Treatment 

AT/AV 

AP 

AP/AS 

Baseline  Treatment 
AT/AV 

Baseline  Treatment 

AT/AV 

AP 

Baseline  Treatment 


-  Level 

AR/AG 

AR/AG 

AR/AG 

AR/AG 

AP/AS 

AP/AS 

AP 

AP 

AP/AS 

AT/AV 


Difference 

0.9028551 

0.8566037 

0.7078222 

0.6095716 

0.2932835 

0.2470321 

0.1950329 

0.1487815 

0.0982506 

0.0462514 


Std  Err  Dif 

0.4355995 

0.4355995 

0.4355995 

0.4355995 

0.4355995 

0.4355995 

0.4355995 

0.4355995 

0.4355995 

0.4355995 


Lower  CL 
0.034079 
-0.012172 
-0.160954 
-0.259204 
-0.575492 
-0.621744 
-0.673743 
-0.719994 
-0.770525 
-0.822524 


Upper  CL 

1.771631 

1.725380 

1.576598 

1.478347 

1.162059 

1.115808 

1.063809 

1.017557 

0.967026 

0.915027 


p-Value 


zi  ▼  Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
zi  Confidence  Quantile 
q*  Alpha 

2.80015  0.05 

^  LSD  Threshold  Matrix 
>  Connecting  Letters  Report 
zi  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Baseline  Treatment  AR/AG 

0.9028551 

0.4355995 

-0.31689 

AT/AV 

AR/AG 

0.8566037 

0.4355995 

-0.36314 

AP 

AR/AG 

0.7078222 

0.4355995 

-0.51192 

AP/AS 

AR/AG 

0.6095716 

0.4355995 

-0.61017 

Baseline  Treatment  AP/AS 

0.2932835 

0.4355995 

-0.92646 

AT/AV 

AP/AS 

0.2470321 

0.4355995 

-0.97271 

Baseline  Treatment  AP 

0.1950329 

0.4355995 

-1.02471 

AT/AV 

AP 

0.1487815 

0.4355995 

-1.07096 

AP 

AP/AS 

0.0982506 

0.4355995 

-1.12149 

Baseline  Treatment  AT/AV 

0.0462514 

0.4355995 

-1.17349 

Upper  CL 

2.122601 

2.076349 

1.927568 

1.829317 

1.513029 

1.466778 

1.414778 

1.368527 

1.317996 

1.265997 


p-Value 


Figure  81:  Future  State  -  (E9)  Add  Experiment  G1  JMP®  Student’s  t  &  Tukey-Kramer  HSD 
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-Oneway  Analysis  of  Flow_days  By  Treatment 


Figure  82:  Future  State  -  (E9)  Add  Experiment  G2  JMP®  ANOVA 

Means  Comparisons 

-  Comparisons  for  each  pair  using  Student's  t 
^  Confidence  Quantile 
t  Alpha 
1.99444  0.05 

>  LSD  Threshold  Matrix 

>  Connecting  Letters  Report 
41  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Upper  CL 

p-Value 

Baseline  Treatment  AR/AG 

1.304606 

0.4245056 

0.457956 

2.151256 

0.0030* 

Baseline  Treatment  AP/AS 

1.240747 

0.4245056 

0.394097 

2.087397 

0.0047* 

AT/AV 

AR/AG 

0.732723 

0.4245056 

-0.113927 

1.579373 

0.0887 

Baseline  Treatment  AP 

0.682916 

0.4245056 

-0.163733 

1.529566 

0.1122 

AT/AV 

AP/AS 

0.668864 

0.4245056 

-0.177786 

1.515514 

0.1196 

AP 

AR/AG 

0.621690 

0.4245056 

-0.224960 

1.468339 

0.1475 

Baseline  Treatment  AT/AV 

0.571883 

0.4245056 

-0.274767 

1.418533 

0.1823 

AP 

AP/AS 

0.557830 

0.4245056 

-0.288819 

1.404480 

0.1931 

AT/AV 

AP 

0.111034 

0.4245056 

-0.735616 

0.957683 

0.7944 

AP/AS 

AR/AG 

0.063859 

0.4245056 

-0.782790 

0.910509 

0.8809 

A  »  Comparisons  for  all  pairs  using  Tukey-Kramer  HSD 
4i  Confidence  Quantile 
q"  Alpha 

2.80015  0.05 

>  LSD  Threshold  Matrix 
D>  Connecting  Letters  Report 
a  Ordered  Differences  Report 


Level 

-  Level 

Difference 

Std  Err  Dif 

Lower  CL 

Upper  CL 

p-Value 

Baseline  Treatment  AR/AG 

1.304606 

0.4245056 

0.11593 

2.493287 

0.0244* 

Baseline  Treatment  AP/AS 

1.240747 

0.4245056 

0.05207 

2.429428 

0.0365* 

AT/AV 

AR/AG 

0.732723 

0.4245056 

-0.45596 

1.921404 

0.4250 

Baseline  Treatment  AP 

0.682916 

0.4245056 

-0.50576 

1.871597 

0.4970 

AT/AV 

AP/AS 

0.668864 

0.4245056 

-0.51982 

1.857545 

0.5178 

AP 

AR/AG 

0.621690 

0.4245056 

-0.56699 

1.810370 

0.5886 

Baseline  Treatment  AT/AV 

0.571883 

0.4245056 

-0.61680 

1.760564 

0.6629 

AP 

AP/AS 

0.557830 

0.4245056 

-0.63085 

1.746511 

0.6835 

AT/AV 

AP 

0.111034 

0.4245056 

-1.07765 

1.299714 

0.9989 

AP/AS 

AR/AG 

0.063859 

0.4245056 

-1.12482 

1.252540 

0.9999 

Figure  83:  Future  State  -  (E9)  Add  Experiment  G2  JMP®  Student’s  t  &  Tukey-Kramer  HSD 
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Figure  84:  Future  State  -  (E9)  Add  Experiment  G3  JMP®  ANOVA 


Means  Comparisons 

A  ▼  Comparisons  for  each  pair  using  Student's  t 
a  Confidence  Quantile 
t  Alpha 
1.99444  0.05 

>  LSD  Threshold  Matrix 
l>  Connecting  Letters  Report 
a  Ordered  Differences  Report 


Level 

Baseline  Treatment 

AR/AG 

AT/AV 

AP 

Baseline  Treatment 
Baseline  Treatment 
AR/AG 
AR/AG 

Baseline  Treatment 
AT/AV 


-  Level 

AP/AS 

AP/AS 

AP/AS 

AP/AS 

AP 

AT/AV 

AP 

AT/AV 

AR/AG 

AP 


Difference 

0.9774360 

0.8182400 

0.6572018 

0.6085417 

0.3688943 

0.3202342 

0.2096984 

0.1610382 

0.1591960 

0.0486602 


Std  Err  Dif 

0.3934269 

0.3934269 

0.3934269 

0.3934269 

0.3934269 

0.3934269 

0.3934269 

0.3934269 

0.3934269 

0.3934269 


Lower  CL 
0.192771 
0.033575 
-0.127463 
-0.176124 
-0.415771 
-0.464431 
-0.574967 
-0.623627 
-0.625469 
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Figure  86:  Future  State  -  (E9)  Add  Experiment  G4  JMP®  ANOVA 
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Figure  88:  Future  State  -  (E9)  Add  Experiment  G5  JMP®  ANOVA 
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Figure  91: 


»  Oneway  Analysis  of  Flow days  By  Treatment 


Figure  90:  Future  State  -  (E9)  Add  Experiment  G6  JMP®  ANOVA 
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Figure  92:  Future  State  -  (E9)  Add  Experiment  G7  JMP®  ANOVA 
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required  from  one  aircraft  to  another  within  each  MDS  and  a  highly  specialized  workforce  that  has  inherent  inflexibility  to  compensate  for  the  workload 
variability.  Air  Force  Materiel  Command  is  reviewing  maintenance  personnel  multi-skilling  as  a  method  to  efficiently  absorb  the  variability  of 
workload  and  maintenance  requirements  between  aircraft. 

This  research  conducts  an  objective  analysis  of  the  F-22  Heavy  Maintenance  Modification  Program  by  building  a  discrete  event  simulation  in 
ARENA  14®  and  performing  a  series  of  designed  experiments.  The  study  analyzes  whether  using  a  multi-skilled  (flexible)  workforce  will  have  an 
impact  on  productivity  of  depot  maintenance  personnel  through  simulation  of  several  multi-skillingpolicies.  The  research  shows  that  multi-skilling 
policies  can  significantly  outperform  overtime-based  production  timelines  at  less  cost,  even  if  individual  skill  proficiencies  decline. 


15.  SUBJECT  TERMS 

Discrete  Event  Simulation,  Flexible  Workforce,  Multi-skilling,  Multiskill,  Multi-trade,  Depot  Maintenance,  Air 
Logistics  Complex 


1  16.  SECURITY  CLASSIFICATION  OF:  1 

17.  LIMITATION  OF 

18.  NUMBER 

19a.  NAMEOF  RESPONSI BLE PERSON 

ABSTRACT 

OF 

AlanW.  Johnson,  PhD,  ENS 

a.  REPORT 

c.  THIS  PAGE 

PAGES 

19b.  TELEPHONE  NUM  BER  (Include  area  code) 

u 

u 

u 

UU 

183 

(937)255-6565,  ext  4703;  e-mail:  alan.johnson@afit.edu 

Standard  Form  298  (Rev.  8-98) 


Prescribed  by  AN  SI  Std.  Z39-1 8 


169 


